JeVois  1.18
JeVois Smart Embedded Machine Vision Toolkit
Share this page:
VideoMapping.H
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #pragma once
19 
21 #include <jevois/Types/Enum.H>
22 
23 #include <iosfwd>
24 #include <vector>
25 #include <string>
26 #include <linux/videodev2.h> // for v4l2_fract
27 
28 namespace jevois
29 {
30  //! Enum for VideoMapping wide-dynamic-range (WDR) type
31  /*! If nothing is specified in a VideoMapping, use None by default. \relates VideoMapping */
32  JEVOIS_DEFINE_ENUM_CLASS(WDRtype, (Linear) (DOL) )
33 
34  //! Enum for VideoMapping crop or rescale
35  /*! On JeVois-Pro Platform (only), the camera ISP can stream up to two images: 1) the raw frame from the sensor
36  (possibly cropped), 2) a scaled frame generated by the ISP (scaling is always applied to the native sensor
37  resolution, irrespective of crop settings), or both. The images can have different pixel types but same
38  frames/s. Getting both cropped and scaled images is useful to use a full-frame raw YUYV capture for GUI display,
39  plus a scaled down RGB24 capture as input to a neural network. When a camera frame size is specified which does
40  not match the sensor's size, either do a centered crop, or a uniform rescaling. Note that rescaling can affect the
41  image aspect ratio, and thus it is recommended that camera frame sizes with same aspect ratio at the native sensor
42  size be used. If nothing is specified in a VideoMapping, use Scale as default. \relates VideoMapping */
43  JEVOIS_DEFINE_ENUM_CLASS(CropType, (Scale) (Crop) (CropScale) )
44 
45  //! Simple struct to hold video mapping definitions for the processing Engine
46  /*! This struct specifies an output video format, resolution, and frame rate (to be send to the end user over USB),
47  the corresponding camera capture video format, resolution and frame rate, and the Module to use to process the
48  camera frames and generate the corresponding output frames. This class also provides conversion functions between
49  frame rate and frame interval periods for both USB and V4L2, which use different units.
50 
51  Operation of JeVois is based on a list of available VideoMapping definitions, which is configured in a file called
52  JEVOIS:config/videomappings.cfg on the microSD card. The video mappings indicate which output formats are exposed
53  to the host computer connected over USB, and which corresponding camera format and vision processing module should
54  be used when a given output format is selected by video capture software running on the host computer.
55 
56  See \ref UserModes for explanations about how to organize videomappings.cfg
57 
58  \ingroup core */
59  struct VideoMapping
60  {
61  unsigned int ofmt; //!< output pixel format, or 0 for no output over USB
62  unsigned int ow; //!< output width
63  unsigned int oh; //!< output height
64  float ofps; //!< output frame rate in frames/sec
65 
66  unsigned int cfmt; //!< camera pixel format
67  unsigned int cw; //!< camera width
68  unsigned int ch; //!< camera height
69  float cfps; //!< camera frame rate in frames/sec
70 
71  unsigned int uvcformat; //!< USB-UVC format number (1-based)
72  unsigned int uvcframe; //!< USB UVC frame number (1-based)
73 
74  std::string vendor; //!< Module creator name, used as a directory to organize the modules
75 
76  std::string modulename; //!< Name of the Module that will process this mapping
77 
78  WDRtype wdr; //!< Type of wide-dynamic-range (WDR) to use, if sensor supports it
79  CropType crop; //!< Type of crop or scaling to apply if camera frame size does not match sensor native
80  bool ispython; //!< True if the module is written in Python; affects behavior of sopath() only
81 
82  unsigned int c2fmt; //!< When crop is CropScale, pixel format of the scaled images, otherwise 0
83  unsigned int c2w; //!< When crop is CropScale, width of the scaled images, otherwise 0
84  unsigned int c2h; //!< When crop is CropScale, height of the scaled images, otherwise 0
85 
86  //! Return the full absolute path and file name of the module's .so or .py file
87  std::string sopath() const;
88 
89  //! Return the full absolute path and file name of the module's .C or .py file
90  std::string srcpath() const;
91 
92  //! Return the size in bytes of an output image
93  unsigned int osize() const;
94 
95  //! Return the size in bytes of a camera image
96  unsigned int csize() const;
97 
98  //! Return the size in bytes of a scaled camera image, if stream==RawAndScaled, otherwise 0
99  unsigned int c2size() const;
100 
101  //! Convert from USB/UVC interval to fps
102  /*! This function rounds to the nearest 1/100 fps. */
103  static float uvcToFps(unsigned int interval);
104 
105  //! Convert from fps to USB/UVC interval
106  static unsigned int fpsToUvc(float fps);
107 
108  //! Convert from V4L2 interval to fps
109  /*! This function rounds to the nearest 1/100 fps. */
110  static float v4l2ToFps(struct v4l2_fract const & interval);
111 
112  //! Convert from fps to V4L2 interval
113  static struct v4l2_fract fpsToV4l2(float fps);
114 
115  //! Return true if this VideoMapping's output format is a match to the given output parameters
116  bool match(unsigned int oformat, unsigned int owidth, unsigned int oheight, float oframespersec) const;
117 
118  //! Return true if camera sensor can support this mapping
119  bool sensorOk(CameraSensor s);
120 
121  //! Convenience function to print out FCC WxH @ fps, for the output (UVC) format
122  std::string ostr() const;
123 
124  //! Convenience function to print out FCC WxH @ fps, for the input (camera) format
125  std::string cstr() const;
126 
127  //! Convenience function to print out FCC WxH @ fps, for the scaled camera input format, if stream==RawAndScaled
128  std::string c2str() const;
129 
130  //! Convenience function to print out FCC WxH @ fps plus possibly second stream, for the input (camera) format
131  std::string cstrall() const;
132 
133  //! Convenience function to print out the whole mapping in a human-friendly way
134  std::string str() const;
135 
136  //! Convenience function to print out the whole mapping in a human-friendly way to be used in a menu
137  std::string menustr() const;
138 
139  //! Equality operator for specs but not vendor or module name
140  /*! Note that two mappings will be declared to match if their fps values are within 0.01fps, to avoid mismatches
141  due to floating point representation and rounding. */
142  bool hasSameSpecsAs(VideoMapping const & other) const;
143 
144  //! Equality operator for specs and also vendor or module name
145  /*! Note that two mappings will be declared to match if their fps values are within 0.01fps, to avoid mismatches
146  due to floating point representation and rounding. */
147  bool isSameAs(VideoMapping const & other) const;
148 
149  //! Determine whether module is C++ or python and set ispython flag accordingly
150  /*! The other fields should have been initialized already. operator>> and loadVideoMappings use this function
151  insternally, so no need to call it after making a VideoMapping from sream. This function throws if neither a
152  .so nor .py file is found in the appropriate place given module vendor and name. */
153  void setModuleType();
154  };
155 
156  //! Stream a VideoMapping out, intended for machines
157  /*! Note that no std::endl is issued at the end. \relates VideoMapping */
158  std::ostream & operator<<(std::ostream & out, VideoMapping const & m);
159 
160  //! Stream a VideoMapping in, intended for machines
161  /*! Note that the assumption is that the mapping is clean (no extra garbage). \relates VideoMapping */
162  std::istream & operator>>(std::istream & in, VideoMapping & m);
163 
164  //! Load all the video mappings from the default config file
165  /*! \relates VideoMapping */
166  std::vector<VideoMapping> loadVideoMappings(CameraSensor s, size_t & defidx, bool checkso = true,
167  bool hasgui = false);
168 
169  //! Parse all the mappings in a config file and also indicate which one is the default
170  /*! The contents of the file are sorted so that the resulting vector is ordered by increasing 1) format fcc, then 2)
171  resolution (from large to small, looking at x first), and 3) framerate (from high to low).
172 
173  The camera format field can have colon-separated prefixes for qualifiers that specify WDR (wide-dynamic-range)
174  camera capture mode and/or crop vs. rescale behavior when camera input dims do not match sensor native dims (only
175  effective on JeVois-Pro).
176 
177  The output width and height can be wither absolute, or relative to camera width and height if prefixed with a + or
178  - symbol.
179 
180  In case of duplicate output formats, frame rates will be decreased by 1fps for each additional duplicate. This is
181  because we need to present the host computer with distinct video formats so that users can select the one they
182  want. For example:
183  \verbatim
184  YUYV 320 240 60.0 YUYV 320 240 60.0 JeVois SaveVideo
185  YUYV 320 240 60.0 YUYV 320 240 60.0 VendorX MyModule
186  YUYV 320 240 60.0 YUYV 320 240 60.0 VendorY MyModule
187  YUYV 320 240 60.0 YUYV 320 240 60.0 VendorZ MyModule
188  \endverbatim
189 
190  will be disambiguated into:
191 
192  \verbatim
193  YUYV 320 240 60.0 YUYV 320 240 60.0 JeVois SaveVideo
194  YUYV 320 240 59.0 YUYV 320 240 60.0 VendorX MyModule
195  YUYV 320 240 58.0 YUYV 320 240 60.0 VendorY MyModule
196  YUYV 320 240 57.0 YUYV 320 240 60.0 VendorZ MyModule
197  \endverbatim
198 
199  and in \b guvcview or similar program running on a host computer, these 4 mappings will be available since they
200  correspond to 4 different framerates. It is recommended that you issue a \b listmapping command in the JeVois
201  command-line interface to confirm the final mappings that are used at runtime after any adjustments; see \ref
202  UserCli for details.
203 
204  defidx is the index of the default format in the resulting vector of mappings. If several default formats are
205  specified, the first one prevails.
206 
207  See \ref UserModes for explanations about how to organize \b videomappings.cfg
208  \relates VideoMapping */
209  std::vector<VideoMapping> videoMappingsFromStream(CameraSensor s, std::istream & is, size_t & defidx,
210  bool checkso, bool hasgui);
211 }
212 
jevois
Definition: Concepts.dox:1
jevois::JEVOIS_DEFINE_ENUM_CLASS
JEVOIS_DEFINE_ENUM_CLASS(CameraSensor,(any)(ov9650)(ov2640)(ov7725)(ar0135)(imx290)(os08a10))
Enum for different sensor models.
jevois::Crop
() Scale() Crop(CropScale)) struct VideoMapping
Simple struct to hold video mapping definitions for the processing Engine.
Definition: VideoMapping.H:43
CameraSensor.H
Enum.H