JeVoisBase  1.3
JeVois Smart Embedded Machine Vision Toolkit Base Modules
Share this page:
RoadNavigation.C
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #include <jevois/Core/Module.H>
19 
20 #include <jevois/Debug/Log.H>
21 #include <jevois/Debug/Timer.H>
24 
25 #include <opencv2/core/core.hpp>
26 #include <opencv2/imgproc/imgproc.hpp>
27 
28 #include <future>
29 #include <linux/videodev2.h> // for v4l2 pixel types
30 
31 // icon by Dave Gandy in transport at flaticon
32 
33 //! Road finder demo
34 /*! This algorithm detects road using a compination of edge detection and tracking, and texture analysis. The algorithm
35  is an implementation of Chang, Siagian and Itti, IROS 2012, available at
36  http://ilab.usc.edu/publications/doc/Chang_etal12iros.pdf
37 
38  The algorithms combines detection and tracking of line segments at the edges of the road or on the road (e.g., lane
39  dividers), and texture analysis to distinguish the road region from its surroundings.
40 
41  The algorithm outputs the horizontal coordinate of the vanishing point of the road, which usually is a good
42  indication of the road heading (except in very tight bends or corners).
43 
44  Demo display outputs
45  --------------------
46 
47  Detected line segments are shown in black and white, while segments that have been reliably tracked over multiple
48  frames are shown in thick purple. Estimated vanishing point location and confidence is shown as a big green disk on
49  the horizon line.
50 
51  Serial Messages
52  ---------------
53 
54  This module can send standardized serial messages as described in \ref UserSerialStyle. One 1D message is issued for
55  on every video frame at vanishing point horizontal location.
56 
57  - Serial message type: \b 1D
58  - `id`: always \b vp (shorthand for vanishing point)
59  - `x`: standardized 2D horizontal coordinate of the filtered vanishing point
60  - `w`: always 0
61  - `extra`: none (empty string)
62 
63  Trying it out
64  -------------
65 
66  To casually try out this module, just search the web for pictures of roads and point the JeVois camera to one of
67  them. Make sure that you align the horizon line of the algorithm (which has a bumber of purple and green disks)
68  roughly with the horizon line in your picture. As you move the camera left and right, the location of the large
69  green disk that marks the detected vanishing point should move left and right, and should point to the vanishing
70  point of the road in your image.
71 
72  When using on a mobile robot in th ereal world, setting the proper horizon line is essential for good operation of
73  the algorithm. This is determined by parameter \c horizon, which should be tuned according to the height and
74  tilt angle of the JeVois camera on your vehicle.
75 
76  @author Laurent Itti
77 
78  @videomapping NONE 0 0 0 YUYV 320 240 30.0 JeVois RoadNavigation
79  @videomapping NONE 0 0 0 YUYV 176 144 120.0 JeVois RoadNavigation
80  @videomapping YUYV 320 256 30.0 YUYV 320 240 30.0 JeVois RoadNavigation
81  @videomapping YUYV 176 160 120.0 YUYV 176 144 120.0 JeVois RoadNavigation
82  @email itti\@usc.edu
83  @address University of Southern California, HNB-07A, 3641 Watt Way, Los Angeles, CA 90089-2520, USA
84  @copyright Copyright (C) 2016 by Laurent Itti, iLab and the University of Southern California
85  @mainurl http://jevois.org
86  @supporturl http://jevois.org/doc
87  @otherurl http://iLab.usc.edu
88  @license GPL v3
89  @distribution Unrestricted
90  @restrictions None
91  \ingroup modules */
93 {
94  public:
95  // ####################################################################################################
96  //! Constructor
97  // ####################################################################################################
98  RoadNavigation(std::string const & instance) :
99  jevois::StdModule(instance), itsProcessingTimer("Processing", 30, LOG_DEBUG)
100  {
101  itsRoadFinder = addSubComponent<RoadFinder>("roadfinder");
102  }
103 
104  // ####################################################################################################
105  //! Virtual destructor for safe inheritance
106  // ####################################################################################################
107  virtual ~RoadNavigation() { }
108 
109  // ####################################################################################################
110  //! Processing function, no video out
111  // ####################################################################################################
112  virtual void process(jevois::InputFrame && inframe) override
113  {
114  // Wait for next available camera image, convert it to gray, and release it:
115  cv::Mat imggray = inframe.getCvGRAY();
116 
117  // Compute the vanishing point, with no drawings:
118  jevois::RawImage visual; // unallocated pixels, will not draw anything
119  itsRoadFinder->process(imggray, visual);
120 
121  // Get the filtered target point x and send to serial:
122  sendSerialImg1Dx(imggray.cols, itsRoadFinder->getFilteredTargetX(), 0.0F, "vp");
123  }
124 
125  // ####################################################################################################
126  //! Processing function with USB video out
127  // ####################################################################################################
128  virtual void process(jevois::InputFrame && inframe, jevois::OutputFrame && outframe) override
129  {
130  // Wait for next available camera image:
131  jevois::RawImage inimg = inframe.get(); unsigned int const w = inimg.width, h = inimg.height;
132  inimg.require("input", w, h, V4L2_PIX_FMT_YUYV); // accept any image size but require YUYV pixels
133 
135 
136  // Convert it to gray:
137  cv::Mat imggray = jevois::rawimage::convertToCvGray(inimg);
138 
139  // Compute the vanishing point. Note: the results will be drawn into inimg, so that we don't have to wait for
140  // outimg to be ready. It's ok to modify the input image, its buffer will be sent back to the camera driver for
141  // capture once we are done here, and will be overwritten anyway:
142  itsRoadFinder->process(imggray, inimg);
143 
144  // Wait for an image from our gadget driver into which we will put our results:
145  jevois::RawImage outimg = outframe.get();
146 
147  // Enforce the correct output image size and format:
148  outimg.require("output", w, h + 16, V4L2_PIX_FMT_YUYV);
149 
150  // Paste the original image + drawings to the top-left corner of the display:
151  unsigned short const txtcol = 0xa0ff; //WHITE: 0x80ff;
152  jevois::rawimage::paste(inimg, outimg, 0, 0);
153  jevois::rawimage::writeText(outimg, "JeVois Road Navigation Demo", 3, 3, txtcol);
154 
155  // Let camera know we are done processing the raw YUV input image:
156  inframe.done();
157 
158  // Clear out the bottom section of the image:
159  jevois::rawimage::drawFilledRect(outimg, 0, h, w, outimg.height-h, jevois::yuyv::Black);
160 
161  // Get the filtered target point x and send to serial:
162  float const tpx = itsRoadFinder->getFilteredTargetX();
163  sendSerialImg1Dx(w, tpx, 0.0F, "vp");
164 
165  // Write some extra info about the vp:
166  std::ostringstream otxt; otxt << std::fixed << std::setprecision(3);
167  auto vp = itsRoadFinder->getCurrVanishingPoint();
168  otxt << "VP x=" << vp.first.i << " (" << vp.second << ") CTR=" << std::setprecision(1);
169  auto cp = itsRoadFinder->getCurrCenterPoint();
170  auto tp = itsRoadFinder->getCurrTargetPoint();
171  otxt << cp.i << " TGT=" << tp.i << " fTPX=" << tpx;
172  jevois::rawimage::writeText(outimg, otxt.str().c_str(), 3, h + 3, jevois::yuyv::White);
173 
174  // Show processing fps:
175  std::string const & fpscpu = itsProcessingTimer.stop();
176  jevois::rawimage::writeText(outimg, fpscpu, 3, h - 13, jevois::yuyv::White);
177 
178  // Send the output image with our processing results to the host over USB:
179  outframe.send();
180  }
181 
182  // ####################################################################################################
183  //! Module internals
184  // ####################################################################################################
185  protected:
187  std::shared_ptr<RoadFinder> itsRoadFinder;
188 };
189 
190 // Allow the module to be loaded as a shared object (.so) file:
RoadNavigation(std::string const &instance)
Constructor.
void writeText(RawImage &img, std::string const &txt, int x, int y, unsigned int col, Font font=Font6x10)
void require(char const *info, unsigned int w, unsigned int h, unsigned int f) const
unsigned int height
virtual void process(jevois::InputFrame &&inframe) override
Processing function, no video out.
std::shared_ptr< RoadFinder > itsRoadFinder
StdModule(std::string const &instance)
jevois::Timer itsProcessingTimer
Module internals.
JEVOIS_REGISTER_MODULE(RoadNavigation)
cv::Mat convertToCvGray(RawImage const &src)
void sendSerialImg1Dx(unsigned int camw, float x, float size=0.0F, std::string const &id="", std::string const &extra="")
void drawFilledRect(RawImage &img, int x, int y, unsigned int w, unsigned int h, unsigned int col)
virtual void process(jevois::InputFrame &&inframe, jevois::OutputFrame &&outframe) override
Processing function with USB video out.
virtual ~RoadNavigation()
Virtual destructor for safe inheritance.
unsigned int width
std::string const & stop()
void paste(RawImage const &src, RawImage &dest, int dx, int dy)
Road finder demo.