JeVoisBase  1.5
JeVois Smart Embedded Machine Vision Toolkit Base Modules
Share this page:
OpticalFlow.C
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #include <jevois/Core/Module.H>
20 
22 
23 #include <linux/videodev2.h>
24 
25 // icon by Gregor Cresnar in arrows at flaticon
26 
27 //! Fast optical flow computation using OF_DIS
28 /*! Computes horizontal and vertical optical flow into two separate optical flow maps. This module is intended for use
29  as a pre-processor, that is, the images sent out may be further processed by the host computer.
30 
31  The optical flow image output is twice taller than the camera input image:
32 
33  - On top is the horizontal motion map
34  + darker shades of gray for things moving rightward from the viewpoint of the camera,
35  + lighter shades for things that move leftward,
36  + mid-grey level for no motion
37 
38  - below it is the vertical motion map
39  + dark for things moving downward,
40  + bright for upward,
41  + mid-grey for no vertical motion.
42 
43  You should be able to run this algorithm at 100 frames/s with 176x144 video camera resolution.
44 
45  This module has parameters to tune the algorithm. Interested users are referred to the original paper and
46  implementation for how to set these:
47 
48  "Fast Optical Flow using Dense Inverse Search" by Till Kroeger, Radu Timofte, Dengxin Dai and Luc Van Gool, Proc
49  ECCV, 2016.
50 
51  Also see here: http://www.vision.ee.ethz.ch/~kroegert/OFlow/
52 
53  Trying it out
54  -------------
55 
56  Have JeVois point towards an otherwise static scene, and swipe one finger in front of it, either moving from left to
57  right of the field of view, from top to bottom, etc to confirm the different greyscale values as described above.
58 
59 
60  @author Laurent Itti
61 
62  @videomapping GREY 176 288 100 YUYV 176 144 100 JeVois OpticalFlow
63  @email itti\@usc.edu
64  @address University of Southern California, HNB-07A, 3641 Watt Way, Los Angeles, CA 90089-2520, USA
65  @copyright Copyright (C) 2016 by Laurent Itti, iLab and the University of Southern California
66  @mainurl http://jevois.org
67  @supporturl http://jevois.org/doc
68  @otherurl http://iLab.usc.edu
69  @license GPL v3
70  @distribution Unrestricted
71  @restrictions None
72  \ingroup modules */
74 {
75  public:
76  //! Constructor
77  OpticalFlow(std::string const & instance) : jevois::Module(instance)
78  { itsOpticalFlow = addSubComponent<FastOpticalFlow>("fastflow"); }
79 
80  //! Virtual destructor for safe inheritance
81  virtual ~OpticalFlow() { }
82 
83  //! Processing function
84  virtual void process(jevois::InputFrame && inframe, jevois::OutputFrame && outframe) override
85  {
86  // Wait for next available camera image:
87  jevois::RawImage inimg = inframe.get(); unsigned int const w = inimg.width, h = inimg.height;
88 
89  // Convert to openCV grayscale:
90  cv::Mat cvimg = jevois::rawimage::convertToCvGray(inimg);
91 
92  // Let camera know we are done processing the input image:
93  inframe.done();
94 
95  // Get the output image and require same width as input, 2x taller, and grey pixels:
96  jevois::RawImage outimg = outframe.get();
97  outimg.require("output", w, h * 2, V4L2_PIX_FMT_GREY);
98 
99  // Process the input and get output:
100  cv::Mat cvout = jevois::rawimage::cvImage(outimg); // cvout's pixels point to outimg's, no copy
101  itsOpticalFlow->process(cvimg, cvout);
102 
103  // Send the output image with our processing results to the host over USB:
104  outframe.send();
105  }
106 
107  private:
108  std::shared_ptr<FastOpticalFlow> itsOpticalFlow;
109 };
110 
111 // Allow the module to be loaded as a shared object (.so) file:
cv::Mat cvImage(RawImage const &src)
friend friend class Module
unsigned int height
virtual void process(jevois::InputFrame &&inframe, jevois::OutputFrame &&outframe) override
Processing function.
Definition: OpticalFlow.C:84
virtual ~OpticalFlow()
Virtual destructor for safe inheritance.
Definition: OpticalFlow.C:81
OpticalFlow(std::string const &instance)
Constructor.
Definition: OpticalFlow.C:77
cv::Mat convertToCvGray(RawImage const &src)
JEVOIS_REGISTER_MODULE(OpticalFlow)
Fast optical flow computation using OF_DIS.
Definition: OpticalFlow.C:73
unsigned int width
void require(char const *info, unsigned int w, unsigned int h, unsigned int f) const