JeVois  1.20
JeVois Smart Embedded Machine Vision Toolkit
Share this page:
RawImageOps.H
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #pragma once
19 
20 #include <jevois/Image/RawImage.H>
21 
22 #include <opencv2/core/core.hpp>
23 
24 namespace jevois
25 {
26  namespace rawimage
27  {
28  //! Create an OpenCV image from the existing RawImage data, sharing the pixel memory rather than copying it
29  /*! The pixel data is not copied, just pointed to, and no pixel format conversion is done. You should destroy the
30  cv::Mat before the RawImage to avoid that your cv::Mat ends up with de-allocated pixel memory. \ingroup image */
31  cv::Mat cvImage(RawImage const & src);
32 
33  //! Convert RawImage to OpenCV doing color conversion from any RawImage source pixel to OpenCV gray byte
34  /*! Supported RawImage pixel formats:
35 
36  - V4L2_PIX_FMT_YUYV
37  - V4L2_PIX_FMT_GREY
38  - V4L2_PIX_FMT_SRGGB8 (Bayer)
39  - V4L2_PIX_FMT_RGB565
40  - V4L2_PIX_FMT_BGR24
41 
42  \ingroup image */
43  cv::Mat convertToCvGray(RawImage const & src);
44 
45  //! Convert RawImage to OpenCV doing color conversion from any RawImage source pixel to OpenCV BGR byte
46  /*! For historical reasons, BGR is the "native" color format of OpenCV, check whether your algorithm needs RGB or
47  BGR and use the appropriate conversion for it.
48 
49  Supported RawImage pixel formats:
50 
51  - V4L2_PIX_FMT_YUYV
52  - V4L2_PIX_FMT_GREY
53  - V4L2_PIX_FMT_SRGGB8 (Bayer)
54  - V4L2_PIX_FMT_RGB565
55  - V4L2_PIX_FMT_BGR24
56 
57  \ingroup image */
58  cv::Mat convertToCvBGR(RawImage const & src);
59 
60  //! Convert RawImage to OpenCV doing color conversion from any RawImage source pixel to OpenCV RGB byte
61  /*! For historical reasons, BGR is the "native" color format of OpenCV, not RGB as created here, check whether your
62  algorithm needs RGB or BGR and use the appropriate conversion for it.
63 
64  Supported RawImage pixel formats:
65  - V4L2_PIX_FMT_YUYV
66  - V4L2_PIX_FMT_GREY
67  - V4L2_PIX_FMT_SRGGB8 (Bayer)
68  - V4L2_PIX_FMT_RGB565
69  - V4L2_PIX_FMT_BGR24
70 
71  \ingroup image */
72  cv::Mat convertToCvRGB(RawImage const & src);
73 
74  //! Convert RawImage to OpenCV doing color conversion from any RawImage source pixel to OpenCV RGB-A byte
75  /*! RGBA is seldom used by OpenCV itself, but is useful for many NEON and OpenGL (GPU) algorithms. For these
76  algorithms, we here just use cv::Mat as a convenient container for raw pixel data.
77 
78  Supported RawImage pixel formats:
79  - V4L2_PIX_FMT_YUYV
80  - V4L2_PIX_FMT_GREY
81  - V4L2_PIX_FMT_SRGGB8 (Bayer)
82  - V4L2_PIX_FMT_RGB565
83  - V4L2_PIX_FMT_BGR24
84 
85  \ingroup image */
86  cv::Mat convertToCvRGBA(RawImage const & src);
87 
88  //! Swap pairs of bytes in a RawImage
89  /*! This should never be needed, except maybe for RGB565 images, mainly for internal debugging, or to directly pass
90  an RGB565 camera image (which is big endian) to USB (which assumes little endian when using RGBP UVC
91  format). This function is NEON-accelerated when compiled for the platform hardware. \ingroup image */
92  void byteSwap(RawImage & img);
93 
94  //! Paste an image within another of same pixel type
95  /*! To keep this function fast, throws if the source image does not fully fit within the destination image, or if
96  the two images do not have the same pixel format. \ingroup image */
97  void paste(RawImage const & src, RawImage & dest, int dx, int dy);
98 
99  //! Paste a grey byte image into a YUYV image
100  /*! \ingroup image */
101  void pasteGreyToYUYV(cv::Mat const & src, RawImage & dest, int dx, int dy);
102 
103  //! Paste a BGR byte image into a YUYV image
104  /*! \ingroup image */
105  void pasteBGRtoYUYV(cv::Mat const & src, RawImage & dst, int dx, int dy);
106 
107  //! Paste a RGB byte image into a YUYV image
108  /*! \ingroup image */
109  void pasteRGBtoYUYV(cv::Mat const & src, RawImage & dst, int dx, int dy);
110 
111  //! Paste an ROI from an image to within another of same pixel type
112  /*! To keep this function fast, throws if the source image does not fully fit within the destination image or if the
113  pixel formats differ. \ingroup image */
114  void roipaste(RawImage const & src, int x, int y, unsigned int w, unsigned int h, RawImage & dest, int dx, int dy);
115 
116  //! Draw a disk in a YUYV image
117  /*! \ingroup image */
118  void drawDisk(RawImage & img, int x, int y, unsigned int rad, unsigned int col);
119 
120  //! Draw a circle in a YUYV image
121  /*! \ingroup image */
122  void drawCircle(RawImage & img, int x, int y, unsigned int rad, unsigned int thick, unsigned int col);
123 
124  //! Clip a line to fit inside a viewport [wxmin...wxmax[ x [wymin...wymax[
125  /*! This uses the Liang-Barsky line clipping algorithm. Returns false if the line is entirely outside the viewport
126  (nothing to draw). \ingroup image */
127  bool clipLine(int wxmin, int wymin, int wxmax, int wymax, int & x1, int & y1, int & x2, int & y2);
128 
129  //! Draw a line in a YUYV image
130  /*! \ingroup image */
131  void drawLine(RawImage & img, int x1, int y1, int x2, int y2, unsigned int thick, unsigned int col);
132 
133  //! Draw a rectangle in a YUYV image
134  /*! The pixels drawn go from (x,y) included to (x+w-1,y+h-1) included. If w or h is 0, it is replaced by 1.
135  \ingroup image */
136  void drawRect(RawImage & img, int x, int y, unsigned int w, unsigned int h, unsigned int thick, unsigned int col);
137 
138  //! Draw a rectangle in a YUYV image
139  /*! This is really not flexible and is optimized for speed only. col will be used for both YU and YV pixels. For
140  more demanding drawings, use openCV functions. The pixels drawn go from (x,y) included to (x+w-1,y+h-1)
141  included. If w or h is 0, it is replaced by 1. \ingroup image */
142  void drawRect(RawImage & img, int x, int y, unsigned int w, unsigned int h, unsigned int col);
143 
144  //! Draw a filled rectangle in a YUYV image
145  /*! This is really not flexible and is optimized for speed only. col will be used for both YU and YV pixels. For
146  more demanding drawings, use openCV functions. The pixels drawn go from (x,y) included to (x+w-1,y+h-1)
147  included. If w or h is 0, it is replaced by 1. \ingroup image */
148  void drawFilledRect(RawImage & img, int x, int y, unsigned int w, unsigned int h, unsigned int col);
149 
150  //! Available fonts for writeText()
151  /*! \ingroup image */
152  enum Font
153  {
154  Font5x7,
155  Font6x10,
156  Font7x13,
157  Font8x13bold,
158  Font9x15bold,
159  Font10x20,
160  Font11x22,
161  Font12x22,
162  Font14x26,
163  Font15x28,
164  Font16x29,
165  Font20x38,
166  };
167 
168  //! Write some text in an image
169  /*! \ingroup image */
170  void writeText(RawImage & img, std::string const & txt, int x, int y, unsigned int col, Font font = Font6x10);
171 
172  //! Write some text in an image
173  /*! \ingroup image */
174  void writeText(RawImage & img, char const * txt, int x, int y, unsigned int col, Font font = Font6x10);
175 
176  //! Convert a BGR cv::Mat to RawImage with already-allocated pixels and pixel type
177  /*! This is used to output video frames over USB from internally computed results in opencv BGR format. dst should
178  already have an allocated pixel buffer (typically, it should be obtained from the USB gadget), and its size
179  must match that of the src image.
180 
181  Supported dst pixel formats:
182  - V4L2_PIX_FMT_YUYV
183  - V4L2_PIX_FMT_GREY
184  - V4L2_PIX_FMT_SRGGB8 (Bayer)
185  - V4L2_PIX_FMT_RGB565
186  - V4L2_PIX_FMT_MJPG
187  - V4L2_PIX_FMT_BGR24
188 
189  quality is used only when dst is MJPG and should be between 1 and 100. \ingroup image */
190  void convertCvBGRtoRawImage(cv::Mat const & src, RawImage & dst, int quality);
191 
192  //! Convert a RGB cv::Mat to RawImage with already-allocated pixels and pixel type
193  /*! This is used to output video frames over USB from internally computed results in opencv BGR format. dst should
194  already have an allocated pixel buffer (typically, it should be obtained from the USB gadget), and its size
195  must match that of the src image.
196 
197  Supported dst pixel formats:
198  - V4L2_PIX_FMT_YUYV
199  - V4L2_PIX_FMT_GREY
200  - V4L2_PIX_FMT_SRGGB8 (Bayer)
201  - V4L2_PIX_FMT_RGB565
202  - V4L2_PIX_FMT_MJPG
203  - V4L2_PIX_FMT_BGR24
204 
205  quality is used only when dst is MJPG and should be between 1 and 100. \ingroup image */
206  void convertCvRGBtoRawImage(cv::Mat const & src, RawImage & dst, int quality);
207 
208  //! Convert an RGBA cv::Mat to RawImage with already-allocated pixels and pixel type
209  /*! This is used to output video frames over USB from internally computed results in opencv RGB-A format (which is
210  often the format of choice for NEON or OpenGL processing). dst should already have an allocated pixel buffer
211  (typically, it should be obtained from the USB gadget), and its size must match that of the src image.
212 
213  Supported dst pixel formats:
214  - V4L2_PIX_FMT_YUYV
215  - V4L2_PIX_FMT_GREY
216  - V4L2_PIX_FMT_SRGGB8 (Bayer)
217  - V4L2_PIX_FMT_RGB565
218  - V4L2_PIX_FMT_MJPG
219  - V4L2_PIX_FMT_BGR24
220 
221  quality is used only when dst is MJPG and should be between 1 and 100. \ingroup image */
222  void convertCvRGBAtoRawImage(cv::Mat const & src, RawImage & dst, int quality);
223 
224  //! Convert a Gray cv::Mat to RawImage with already-allocated pixels and pixel type
225  /*! This is used to output video frames over USB from internally computed results in opencv Gray format. dst should
226  already have an allocated pixel buffer (typically, it should be obtained from the USB gadget), and its size
227  must match that of the src image.
228 
229  Supported dst pixel formats:
230  - V4L2_PIX_FMT_YUYV
231  - V4L2_PIX_FMT_GREY
232  - V4L2_PIX_FMT_SRGGB8 (Bayer)
233  - V4L2_PIX_FMT_RGB565
234  - V4L2_PIX_FMT_MJPG
235  - V4L2_PIX_FMT_BGR24
236 
237  quality is used only when dst is MJPG and should be between 1 and 100. \ingroup image */
238  void convertCvGRAYtoRawImage(cv::Mat const & src, RawImage & dst, int quality);
239 
240  //! Split an RGBA cv::Mat into a 4x taller grey RawImage with already-allocated pixels
241  /*! This is used to run GPU shaders that apply different filters to an image and store their results separately into
242  the R, G, B and A components of an RGBA output image. Here, we just unpack the 4 components and stuff them into
243  the output image one above the other: R on top, then G below it, then B below it, then A below it. The
244  destination RawImage must be at least as wide as src and at least 4 times as tall as src, and must have
245  V4L2_PIX_FMT_GREY pixels. \ingroup image */
246  void unpackCvRGBAtoGrayRawImage(cv::Mat const & src, RawImage & dst);
247 
248  //! Flip a YUYV RawImage horizontally while preserving color information
249  /*! This function is to allow one to use JeVois with the PhotoBooth app on a Mac, whih flips the image horizontally
250  (about the vertical axis). Only YUYV pixels are supported. You might be able to use cv::flip() from OpenCV
251  instead with other pixel types. \ingroup image */
252  void hFlipYUYV(RawImage & img);
253 
254  //! OpenCV does not provide conversion from RGB to YUYV in cvtColor(), so this function provides it
255  /*! \ingroup image */
256  void convertCvRGBtoCvYUYV(cv::Mat const & src, cv::Mat & dst);
257 
258  //! OpenCV does not provide conversion from BGR to YUYV in cvtColor(), so this function provides it
259  /*! \ingroup image */
260  void convertCvBGRtoCvYUYV(cv::Mat const & src, cv::Mat & dst);
261 
262  //! OpenCV does not provide conversion from GRAY to YUYV in cvtColor(), so this function provides it
263  /*! \ingroup image */
264  void convertCvGRAYtoCvYUYV(cv::Mat const & src, cv::Mat & dst);
265 
266  //! OpenCV does not provide conversion from RGBA to YUYV in cvtColor(), so this function provides it
267  /*! \ingroup image */
268  void convertCvRGBAtoCvYUYV(cv::Mat const & src, cv::Mat & dst);
269 
270  //! Convert from Bayer to YUYV, only used internally by Camera class
271  /*! Both src and dst should already have allocated buffers of correct sizes. */
272  void convertBayerToYUYV(RawImage const & src, RawImage & dst);
273 
274  //! Convert from Grey (monochrome) to YUYV, only used internally by Camera class
275  /*! Both src and dst should already have allocated buffers of correct sizes. */
276  void convertGreyToYUYV(RawImage const & src, RawImage & dst);
277 
278  } // namespace rawimage
279 
280  //! Rescale an OpenCV image, choosing the right kind of interpolation
281  /*! \ingroup image */
282  cv::Mat rescaleCv(cv::Mat const & img, cv::Size const & newdims);
283 
284 } // namespace jevois
285 
jevois::rawimage::convertBayerToYUYV
void convertBayerToYUYV(RawImage const &src, RawImage &dst)
Convert from Bayer to YUYV, only used internally by Camera class.
Definition: RawImageOps.C:1593
jevois::rawimage::Font15x28
@ Font15x28
Definition: RawImageOps.H:163
jevois::rawimage::drawDisk
void drawDisk(RawImage &img, int x, int y, unsigned int rad, unsigned int col)
Draw a disk in a YUYV image.
Definition: RawImageOps.C:465
jevois::rawimage::convertCvRGBAtoCvYUYV
void convertCvRGBAtoCvYUYV(cv::Mat const &src, cv::Mat &dst)
OpenCV does not provide conversion from RGBA to YUYV in cvtColor(), so this function provides it.
Definition: RawImageOps.C:1256
jevois::rawimage::Font20x38
@ Font20x38
Definition: RawImageOps.H:165
jevois::rawimage::Font9x15bold
@ Font9x15bold
Definition: RawImageOps.H:158
jevois::rawimage::convertCvRGBtoCvYUYV
void convertCvRGBtoCvYUYV(cv::Mat const &src, cv::Mat &dst)
OpenCV does not provide conversion from RGB to YUYV in cvtColor(), so this function provides it.
Definition: RawImageOps.C:1112
jevois::rescaleCv
cv::Mat rescaleCv(cv::Mat const &img, cv::Size const &newdims)
Rescale an OpenCV image, choosing the right kind of interpolation.
Definition: RawImageOps.C:1624
jevois::rawimage::convertCvRGBtoRawImage
void convertCvRGBtoRawImage(cv::Mat const &src, RawImage &dst, int quality)
Convert a RGB cv::Mat to RawImage with already-allocated pixels and pixel type.
Definition: RawImageOps.C:1289
jevois::rawimage::Font14x26
@ Font14x26
Definition: RawImageOps.H:162
RawImage.H
jevois::rawimage::convertToCvRGB
cv::Mat convertToCvRGB(RawImage const &src)
Convert RawImage to OpenCV doing color conversion from any RawImage source pixel to OpenCV RGB byte.
Definition: RawImageOps.C:307
jevois::rawimage::Font
Font
Available fonts for writeText()
Definition: RawImageOps.H:152
jevois::RawImage
A raw image as coming from a V4L2 Camera and/or being sent out to a USB Gadget.
Definition: RawImage.H:110
jevois::rawimage::drawCircle
void drawCircle(RawImage &img, int x, int y, unsigned int rad, unsigned int thick, unsigned int col)
Draw a circle in a YUYV image.
Definition: RawImageOps.C:483
jevois::rawimage::convertToCvBGR
cv::Mat convertToCvBGR(RawImage const &src)
Convert RawImage to OpenCV doing color conversion from any RawImage source pixel to OpenCV BGR byte.
Definition: RawImageOps.C:282
jevois::rawimage::Font6x10
@ Font6x10
Definition: RawImageOps.H:155
jevois::rawimage::convertToCvGray
cv::Mat convertToCvGray(RawImage const &src)
Convert RawImage to OpenCV doing color conversion from any RawImage source pixel to OpenCV gray byte.
Definition: RawImageOps.C:246
jevois::rawimage::roipaste
void roipaste(RawImage const &src, int x, int y, unsigned int w, unsigned int h, RawImage &dest, int dx, int dy)
Paste an ROI from an image to within another of same pixel type.
Definition: RawImageOps.C:425
jevois::rawimage::writeText
void writeText(RawImage &img, std::string const &txt, int x, int y, unsigned int col, Font font=Font6x10)
Write some text in an image.
Definition: RawImageOps.C:689
jevois::rawimage::convertCvGRAYtoRawImage
void convertCvGRAYtoRawImage(cv::Mat const &src, RawImage &dst, int quality)
Convert a Gray cv::Mat to RawImage with already-allocated pixels and pixel type.
Definition: RawImageOps.C:1359
jevois
Definition: Concepts.dox:1
jevois::rawimage::Font8x13bold
@ Font8x13bold
Definition: RawImageOps.H:157
jevois::rawimage::Font12x22
@ Font12x22
Definition: RawImageOps.H:161
jevois::rawimage::Font5x7
@ Font5x7
Definition: RawImageOps.H:154
jevois::rawimage::pasteRGBtoYUYV
void pasteRGBtoYUYV(cv::Mat const &src, RawImage &dst, int dx, int dy)
Paste a RGB byte image into a YUYV image.
Definition: RawImageOps.C:1122
jevois::rawimage::drawFilledRect
void drawFilledRect(RawImage &img, int x, int y, unsigned int w, unsigned int h, unsigned int col)
Draw a filled rectangle in a YUYV image.
Definition: RawImageOps.C:646
jevois::rawimage::hFlipYUYV
void hFlipYUYV(RawImage &img)
Flip a YUYV RawImage horizontally while preserving color information.
Definition: RawImageOps.C:1416
jevois::rawimage::convertCvRGBAtoRawImage
void convertCvRGBAtoRawImage(cv::Mat const &src, RawImage &dst, int quality)
Convert an RGBA cv::Mat to RawImage with already-allocated pixels and pixel type.
Definition: RawImageOps.C:1337
jevois::rawimage::clipLine
bool clipLine(int wxmin, int wymin, int wxmax, int wymax, int &x1, int &y1, int &x2, int &y2)
Clip a line to fit inside a viewport [wxmin...wxmax[ x [wymin...wymax[.
Definition: RawImageOps.C:533
jevois::rawimage::convertCvBGRtoRawImage
void convertCvBGRtoRawImage(cv::Mat const &src, RawImage &dst, int quality)
Convert a BGR cv::Mat to RawImage with already-allocated pixels and pixel type.
Definition: RawImageOps.C:1266
jevois::rawimage::unpackCvRGBAtoGrayRawImage
void unpackCvRGBAtoGrayRawImage(cv::Mat const &src, RawImage &dst)
Split an RGBA cv::Mat into a 4x taller grey RawImage with already-allocated pixels.
Definition: RawImageOps.C:1312
jevois::rawimage::Font16x29
@ Font16x29
Definition: RawImageOps.H:164
jevois::rawimage::Font11x22
@ Font11x22
Definition: RawImageOps.H:160
jevois::rawimage::cvImage
cv::Mat cvImage(RawImage const &src)
Create an OpenCV image from the existing RawImage data, sharing the pixel memory rather than copying ...
Definition: RawImageOps.C:31
jevois::rawimage::pasteBGRtoYUYV
void pasteBGRtoYUYV(cv::Mat const &src, RawImage &dst, int dx, int dy)
Paste a BGR byte image into a YUYV image.
Definition: RawImageOps.C:1036
jevois::rawimage::paste
void paste(RawImage const &src, RawImage &dest, int dx, int dy)
Paste an image within another of same pixel type.
Definition: RawImageOps.C:403
jevois::rawimage::drawRect
void drawRect(RawImage &img, int x, int y, unsigned int w, unsigned int h, unsigned int thick, unsigned int col)
Draw a rectangle in a YUYV image.
Definition: RawImageOps.C:607
jevois::rawimage::convertToCvRGBA
cv::Mat convertToCvRGBA(RawImage const &src)
Convert RawImage to OpenCV doing color conversion from any RawImage source pixel to OpenCV RGB-A byte...
Definition: RawImageOps.C:332
h
int h
Definition: GUIhelper.C:2373
jevois::rawimage::convertCvBGRtoCvYUYV
void convertCvBGRtoCvYUYV(cv::Mat const &src, cv::Mat &dst)
OpenCV does not provide conversion from BGR to YUYV in cvtColor(), so this function provides it.
Definition: RawImageOps.C:1026
jevois::rawimage::Font7x13
@ Font7x13
Definition: RawImageOps.H:156
jevois::rawimage::drawLine
void drawLine(RawImage &img, int x1, int y1, int x2, int y2, unsigned int thick, unsigned int col)
Draw a line in a YUYV image.
Definition: RawImageOps.C:564
jevois::rawimage::Font10x20
@ Font10x20
Definition: RawImageOps.H:159
jevois::rawimage::pasteGreyToYUYV
void pasteGreyToYUYV(cv::Mat const &src, RawImage &dest, int dx, int dy)
Paste a grey byte image into a YUYV image.
Definition: RawImageOps.C:448
jevois::rawimage::convertGreyToYUYV
void convertGreyToYUYV(RawImage const &src, RawImage &dst)
Convert from Grey (monochrome) to YUYV, only used internally by Camera class.
Definition: RawImageOps.C:1613
jevois::rawimage::byteSwap
void byteSwap(RawImage &img)
Swap pairs of bytes in a RawImage.
Definition: RawImageOps.C:368
jevois::rawimage::convertCvGRAYtoCvYUYV
void convertCvGRAYtoCvYUYV(cv::Mat const &src, cv::Mat &dst)
OpenCV does not provide conversion from GRAY to YUYV in cvtColor(), so this function provides it.
Definition: RawImageOps.C:1184