JeVois  1.17
JeVois Smart Embedded Machine Vision Toolkit
Share this page:
OutputFrame.H
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #pragma once
19 
20 #include <jevois/Image/RawImage.H>
21 #include <opencv2/core/core.hpp>
22 #include <memory>
23 
24 namespace jevois
25 {
26  class VideoOutput;
27  class Engine;
28 
29  //! Exception-safe wrapper around a raw image to be sent over USB
30  /*! This wrapper operates much like std:future in standard C++11. Users can get the next memory-allocated but blank
31  image to be sent over USB by calling get(), which may block if all buffers are still being sent over USB by Gadget
32  and no blank one is available, or may throw if getting that buffer fails for some reason (e.g., usb disconnect,
33  user just changed video mode in their webcam software or closed it). The allocated image size and pixel type is as
34  defined by the current VideoMapping, USB section, i.e., it is the USB video mode currently selected by the
35  user. To save time, image buffers are not zeroed out, so you should not assume that the image is filled with black
36  pixels, it could contain random pixels, or previous output frames. In addition, a send() function is provided
37  which users may use as soon as they are finished with writing the pixel data into the image obtained via get(), to
38  allow the USB driver to send that image to the connected host computer. If send() has not been called by the time
39  the OutputFrame is destroyed, it will be called automatically, if get() had been called.
40 
41  OutputFrame implements a zero-copy, zero-wait access to output video frames, that is:
42 
43  1. the pixel data of the image you obtain via get() is directly the memory-mapped pixel buffer that the silicon
44  hardware on the JeVois chip will use via direct-memory-access (DMA) to stream the data out over the USB link;
45  2. as soon as you call send() that buffer will be queued for sending over USB (as opposed to having a fixed,
46  regular interval at which images may be streamed out). Gadget has several image buffers, allowing one to be
47  streamed out over USB while another is being handed over for filling by your application via get(). These
48  buffers are recycled, i.e., once send() is called, the underlying buffer is streamed over USB and then sent
49  back to the Gadget for future access by your code.
50 
51  \ingroup core */
53  {
54  public:
55  //! Move constructor
56  OutputFrame(OutputFrame && other) = default;
57 
58  //! Get a pre-allocated image so that we can fill the pixel data and later send out over USB using send()
59  /*! May throw if not buffer is available, i.e., all have been queued to send to the host but have not yet been
60  sent. Application code must balance exactly one send() for each get(). */
61  RawImage const & get() const;
62 
63  //! Send an image out over USB to the host computer
64  /*! May throw if the format is incorrect or std::overflow_error if we have not yet consumed the previous image. */
65  void send() const;
66 
67  //! Shorthand to send a cv::Mat after converting / scaling it to the current output format
68  /*! This is mostly intended for Python module writers, as they will likely use OpenCV for all their image
69  processing. The cv::Mat will be rescaled to the same dims as the output frame.
70 
71  The pixel format of the given cv::Mat is guessed as follows:
72  - if img.type() == CV_8UC3, assume BGR pixels
73  - if img.type() == CV_8UC1, assume GRAY pixels
74  - if img.type() == CV_8UC4, assume RGBA pixels
75 
76  C++ module writers should stick to the get()/send() pair as this provides better fine-grained control. Note
77  that the cv::Mat image will always be copied or converted to the destination RawImage and will then be sent
78  out immediately by calling send(), so users should not call send() after using this function. This function is
79  basically equivalent to calling get(), converting the given cv::Mat to the proper output format, and calling
80  send(). quality is used only if the output format is MJPEG and should be between 1 and 100.*/
81  void sendCv(cv::Mat const & img, int quality = 75) const;
82 
83  //! Shorthand to send a GRAY cv::Mat after converting it to the current output format
84  /*! This is mostly intended for Python module writers, as they will likely use OpenCV for all their image
85  processing. The cv::Mat must have same dims as the output frame. C++ module writers should stick to the
86  get()/send() pair as this provides better fine-grained control. Note that the cv::Mat image will always be
87  copied or converted to the destination RawImage and will then be sent out immediately by calling send(), so
88  users should not call send() after using this function. This function is basically equivalent to calling
89  get(), converting the given cv::Mat to the proper output format, and calling send(). quality is used only if
90  the output format is MJPEG and should be between 1 and 100. */
91  void sendCvGRAY(cv::Mat const & img, int quality = 75) const;
92 
93  //! Shorthand to send a BGR cv::Mat after converting it to the current output format
94  /*! This is mostly intended for Python module writers, as they will likely use OpenCV for all their image
95  processing. The cv::Mat must have same dims as the output frame. C++ module writers should stick to the
96  get()/send() pair as this provides better fine-grained control. Note that the cv::Mat image will always be
97  copied or converted to the destination RawImage and will then be sent out immediately by calling send(), so
98  users should not call send() after using this function. This function is basically equivalent to calling
99  get(), converting the given cv::Mat to the proper output format, and calling send(). quality is used only if
100  the output format is MJPEG and should be between 1 and 100.*/
101  void sendCvBGR(cv::Mat const & img, int quality = 75) const;
102 
103  //! Shorthand to send a RGB cv::Mat after converting it to the current output format
104  /*! This is mostly intended for Python module writers, as they will likely use OpenCV for all their image
105  processing. The cv::Mat must have same dims as the output frame. C++ module writers should stick to the
106  get()/send() pair as this provides better fine-grained control. Note that the cv::Mat image will always be
107  copied or converted to the destination RawImage and will then be sent out immediately by calling send(), so
108  users should not call send() after using this function. This function is basically equivalent to calling
109  get(), converting the given cv::Mat to the proper output format, and calling send(). quality is used only if
110  the output format is MJPEG and should be between 1 and 100.*/
111  void sendCvRGB(cv::Mat const & img, int quality = 75) const;
112 
113  //! Shorthand to send a RGBA cv::Mat after converting it to the current output format
114  /*! This is mostly intended for Python module writers, as they will likely use OpenCV for all their image
115  processing. The cv::Mat must have same dims as the output frame. C++ module writers should stick to the
116  get()/send() pair as this provides better fine-grained control. Note that the cv::Mat image will always be
117  copied or converted to the destination RawImage and will then be sent out immediately by calling send(), so
118  users should not call send() after using this function. This function is basically equivalent to calling
119  get(), converting the given cv::Mat to the proper output format, and calling send(). quality is used only if
120  the output format is MJPEG and should be between 1 and 100.*/
121  void sendCvRGBA(cv::Mat const & img, int quality = 75) const;
122 
123  //! Shorthand to send a GRAY cv::Mat after converting it to the current output format
124  /*! This is mostly intended for Python module writers, as they will likely use OpenCV for all their image
125  processing. The cv::Mat will be rescaled to the same dims as the output frame. C++ module writers should stick
126  to the get()/send() pair as this provides better fine-grained control. Note that the cv::Mat image will always
127  be copied or converted to the destination RawImage and will then be sent out immediately by calling send(), so
128  users should not call send() after using this function. This function is basically equivalent to calling
129  get(), converting the given cv::Mat to the proper output format, and calling send(). quality is used only if
130  the output format is MJPEG and should be between 1 and 100. */
131  void sendScaledCvGRAY(cv::Mat const & img, int quality = 75) const;
132 
133  //! Shorthand to send a BGR cv::Mat after converting it to the current output format
134  /*! This is mostly intended for Python module writers, as they will likely use OpenCV for all their image
135  processing. The cv::Mat will be rescaled to the same dims as the output frame. C++ module writers should stick
136  to the get()/send() pair as this provides better fine-grained control. Note that the cv::Mat image will always
137  be copied or converted to the destination RawImage and will then be sent out immediately by calling send(), so
138  users should not call send() after using this function. This function is basically equivalent to calling
139  get(), converting the given cv::Mat to the proper output format, and calling send(). quality is used only if
140  the output format is MJPEG and should be between 1 and 100.*/
141  void sendScaledCvBGR(cv::Mat const & img, int quality = 75) const;
142 
143  //! Shorthand to send a RGB cv::Mat after converting it to the current output format
144  /*! This is mostly intended for Python module writers, as they will likely use OpenCV for all their image
145  processing. The cv::Mat will be rescaled to the same dims as the output frame. C++ module writers should stick
146  to the get()/send() pair as this provides better fine-grained control. Note that the cv::Mat image will always
147  be copied or converted to the destination RawImage and will then be sent out immediately by calling send(), so
148  users should not call send() after using this function. This function is basically equivalent to calling
149  get(), converting the given cv::Mat to the proper output format, and calling send(). quality is used only if
150  the output format is MJPEG and should be between 1 and 100.*/
151  void sendScaledCvRGB(cv::Mat const & img, int quality = 75) const;
152 
153  //! Shorthand to send a RGBA cv::Mat after converting it to the current output format
154  /*! This is mostly intended for Python module writers, as they will likely use OpenCV for all their image
155  processing. The cv::Mat will be rescaled to the same dims as the output frame. C++ module writers should stick
156  to the get()/send() pair as this provides better fine-grained control. Note that the cv::Mat image will always
157  be copied or converted to the destination RawImage and will then be sent out immediately by calling send(), so
158  users should not call send() after using this function. This function is basically equivalent to calling
159  get(), converting the given cv::Mat to the proper output format, and calling send(). quality is used only if
160  the output format is MJPEG and should be between 1 and 100.*/
161  void sendScaledCvRGBA(cv::Mat const & img, int quality = 75) const;
162 
163  //! Destructor, returns the buffers to the driver as needed
164  ~OutputFrame();
165 
166  private:
167  OutputFrame() = delete;
168  OutputFrame(OutputFrame const & other) = delete;
169  OutputFrame & operator=(OutputFrame const & other) = delete;
170 
171  // Only our friends can construct us:
172  friend class Engine;
173  OutputFrame(std::shared_ptr<VideoOutput> const & gad, RawImage * excimg = nullptr);
174 
175  std::shared_ptr<VideoOutput> itsGadget;
176  mutable bool itsDidGet;
177  mutable bool itsDidSend;
178  mutable RawImage itsImage;
179  jevois::RawImage * itsImagePtrForException;
180  };
181 
182 } // namespace jevois
jevois::OutputFrame::sendCvGRAY
void sendCvGRAY(cv::Mat const &img, int quality=75) const
Shorthand to send a GRAY cv::Mat after converting it to the current output format.
Definition: OutputFrame.C:83
jevois::OutputFrame
Exception-safe wrapper around a raw image to be sent over USB.
Definition: OutputFrame.H:52
jevois::OutputFrame::sendScaledCvRGBA
void sendScaledCvRGBA(cv::Mat const &img, int quality=75) const
Shorthand to send a RGBA cv::Mat after converting it to the current output format.
Definition: OutputFrame.C:140
RawImage.H
jevois::RawImage
A raw image as coming from a V4L2 Camera and/or being sent out to a USB Gadget.
Definition: RawImage.H:110
jevois::OutputFrame::sendScaledCvGRAY
void sendScaledCvGRAY(cv::Mat const &img, int quality=75) const
Shorthand to send a GRAY cv::Mat after converting it to the current output format.
Definition: OutputFrame.C:114
jevois::OutputFrame::~OutputFrame
~OutputFrame()
Destructor, returns the buffers to the driver as needed.
Definition: OutputFrame.C:31
jevois
Definition: Concepts.dox:1
jevois::Engine
JeVois processing engine - gets images from camera sensor, processes them, and sends results over USB...
Definition: Engine.H:381
jevois::OutputFrame::sendCvRGB
void sendCvRGB(cv::Mat const &img, int quality=75) const
Shorthand to send a RGB cv::Mat after converting it to the current output format.
Definition: OutputFrame.C:98
jevois::OutputFrame::sendCv
void sendCv(cv::Mat const &img, int quality=75) const
Shorthand to send a cv::Mat after converting / scaling it to the current output format.
Definition: OutputFrame.C:71
jevois::OutputFrame::sendScaledCvRGB
void sendScaledCvRGB(cv::Mat const &img, int quality=75) const
Shorthand to send a RGB cv::Mat after converting it to the current output format.
Definition: OutputFrame.C:131
jevois::OutputFrame::sendCvRGBA
void sendCvRGBA(cv::Mat const &img, int quality=75) const
Shorthand to send a RGBA cv::Mat after converting it to the current output format.
Definition: OutputFrame.C:106
jevois::OutputFrame::send
void send() const
Send an image out over USB to the host computer.
Definition: OutputFrame.C:63
jevois::OutputFrame::sendCvBGR
void sendCvBGR(cv::Mat const &img, int quality=75) const
Shorthand to send a BGR cv::Mat after converting it to the current output format.
Definition: OutputFrame.C:91
jevois::OutputFrame::get
const RawImage & get() const
Get a pre-allocated image so that we can fill the pixel data and later send out over USB using send()
Definition: OutputFrame.C:55
jevois::OutputFrame::sendScaledCvBGR
void sendScaledCvBGR(cv::Mat const &img, int quality=75) const
Shorthand to send a BGR cv::Mat after converting it to the current output format.
Definition: OutputFrame.C:123