JeVois  1.18
JeVois Smart Embedded Machine Vision Toolkit
Share this page:
Camera.H
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #pragma once
19 
20 #include <jevois/Core/VideoInput.H>
23 
24 #include <mutex>
25 
26 namespace jevois
27 {
28  //! JeVois camera driver class - grabs frames from a Video4Linux camera sensor
29  /*! On the platform hardware, the Camera class provides access to the camera sensor on the parallel camera bus
30  (CSI). On other hardware, it can provide access to any Video4Linux2 camera (e.g., USB webcam), although with some
31  limitations, such as:
32  - cannot read/write raw camera registers.
33  - usually cannot select any frames/s but instead limited to a few fixed values like 15fps, 30fps.
34  - usually limited to 3fps max (no 60fps or 120fps modes like on JeVois platform hardware)
35  - usually limited number of controls (exposure, gain, etc) available.
36 
37  The camera runs a thread that handles capturing frames automatically. Users can access the next grabbed frame
38  using get(), which may block until the frame is fully captured. Once they are finished with a frame obtained
39  through get(), users should hand it back to the Camera by calling done(). When using the JeVois Engine and a
40  Module, get() and done() are automatically called in an exception-safe manner through the use of the InputFrame
41  class. Indeed, video grabbing uses a fixed set of image buffers that are allocated by the Linux kernel when a
42  video format is configured, and the camera hardware then cycles through the buffers. On the platform, the camera
43  image buffers are allocated in the ION carveout memory area, which is a reserved memory zone configured in the
44  Linux kernel, and the USB output buffers are also allocated in that zone. To avoid reserving too much memory that
45  will not be used, the ION carveout reservation is quite small by default, but is sufficient for 3 1280x1024 YUYV
46  buffers.
47 
48  Camera implements a zero-copy, zero-wait access to input video frames, that is:
49  - the pixel data of the image you obtain via get() is directly the memory-mapped pixel buffer that the silicon
50  hardware on the JeVois chip uses via direct-memory-access (DMA) to stream the pixel data from the camera chip to
51  processor memory;
52  - as soon as an image is captured by the camera hardware, get() unblocks and returns it (as opposed to having a
53  fixed, regular interval at which images may be available). Camera has several image buffers, allowing one to be
54  captured while another is being handed over for processing via get(). These buffers are recycled, i.e., once
55  done() is called, the underlying buffer is sent back to the camera hardware for future capture.
56 
57  Camera will drop frames if they are not processed by user code fast enough. That is, if all buffers have been
58  grabbed already and not yet handed over through get() and then returned by done(), then the next camera sensor
59  frame will be dropped.
60 
61  Most programmers will never use Camera directly, instead using Engine and InputFrame. \ingroup core */
62  class Camera : public VideoInput
63  {
64  public:
65  //! Construct and open the device
66  /*! \param devname device name, e.g., /dev/video0
67  \param nbufs number of video grab buffers, or 0 for automatic. */
68  Camera(std::string const & devname, jevois::CameraSensor s = jevois::CameraSensor::any,
69  unsigned int const nbufs = 0);
70 
71  //! Close the device and free all resources
72  ~Camera();
73 
74  //! Start streaming
75  void streamOn() override;
76 
77  //! Abort streaming
78  /*! This only cancels future get() and done() calls, one should still call streamOff() to turn off streaming. */
79  void abortStream() override;
80 
81  //! Stop streaming
82  void streamOff() override;
83 
84  //! Get the next captured buffer
85  /*! Throws if we are not streaming or blocks until an image is available (has been captured). img should have been
86  created by the caller (with no pixel buffer allocated) and will be filled in by what we receive from the
87  device here. */
88  void get(RawImage & img) override;
89 
90  //! Check whether a second input image scaled by the JeVoisPro Platform ISP is available
91  /*! Returns false unless we are on JeVois-Pro Platform and the camera format modifier jevois::CropType::CropScale
92  is currently in use. */
93  bool hasScaledImage() const override;
94 
95  //! Get the next captured buffer, for second ISP-scaled image
96  /*! On JeVois-Pro Platform only, the camera ISP can output 2 frames: 1) raw from sensor, 2) scaled by ISP. This
97  function is to access the ISP scaled frame. Throws if not JeVois-Pro Platform or the camera stream type is not
98  jevois::StreamType::RawAndScaled. Throws if we are not streaming or blocks until an image is available (has
99  been captured). img should have been created by the caller (with no pixel buffer allocated) and will be filled
100  in by what we receive from the device here. */
101  void get2(RawImage & img) override;
102 
103  //! Indicate that user processing is done with an image previously obtained via get()
104  /*! You should call this as soon after get() as possible, once you are finished with the RawImage data so that it
105  can be recycled.
106 
107  Calling done() on a RawImage invalidates the image and in particular its pixel buffer. Users should make sure
108  that no attempt to use the image or the pixel buffer will be made after done() is called. */
109  void done(RawImage & img) override;
110 
111  //! Indicate that user processing is done with an image previously obtained via get2()
112  /*! You should call this as soon after get2() as possible, once you are finished with the RawImage data so that it
113  can be recycled.
114 
115  Calling done2() on a RawImage invalidates the image and in particular its pixel buffer. Users should make sure
116  that no attempt to use the image or the pixel buffer will be made after done2() is called. */
117  void done2(RawImage & img) override;
118 
119  //! Get information about a control, throw if unsupported by hardware
120  /*! Caller should zero-out qc and then set the id field to the desired control id. See VIDIOC_QUERYCTRL for more
121  information. */
122  void queryControl(struct v4l2_queryctrl & qc) const override;
123 
124  //! Get the available menu entry names for a menu-type control, throw if unsupported by hardware
125  /*! Caller should zero-out qm and then set the id and index fields to the desired control id and menu item
126  index. See VIDIOC_QUERYMENU for more information. */
127  void queryMenu(struct v4l2_querymenu & qm) const override;
128 
129  //! Get a control's current value, throw if unsupported by hardware
130  /*! This is just a pass-through to VIDIOC_G_CTRL, users should zero-out ctrl and then set in ctrl.id the desired
131  control ID. */
132  void getControl(struct v4l2_control & ctrl) const;
133 
134  //! Set a control, throw if control not supported or the hardware rejects the value
135  /*! This is just a pass-through to VIDIOC_S_CTRL */
136  void setControl(struct v4l2_control const & ctrl) override;
137 
138  //! Set the video format and frame rate
139  void setFormat(VideoMapping const & m) override;
140 
141  //! Write a value to one of the camera's registers
142  /*! This very low-level access is for development of optimal camera settings only and should not be used in normal
143  operation, it can crash your system. */
144  void writeRegister(unsigned short reg, unsigned short val);
145 
146  //! Read a value from one of the camera's registers
147  /*! This very low-level access is for development of optimal camera settings only and should not be used in normal
148  operation, it can crash your system. */
149  unsigned short readRegister(unsigned short reg);
150 
151  //! Lock the camera and return its file descriptor
152  /*! Used by IMUi2c to access the IMU registers over the I2C bus shared with the camera sensor. Use with caution
153  and make sure you catch exceptions so you can guarantee that you will call unlock(), otherwise your camera
154  will be stuck and enable to stream. */
155  int lock();
156 
157  //! Unlock the camera that was previously locked by lock()
158  void unlock();
159 
160  protected:
161  //! Sensor flags
162  /*! Keep this in sync with jevois-sdk/linux-3.4/drivers/media/video/sunxi-vfe/device/camera.h */
163  enum Flags
164  {
168  };
169 
170  //! Get the sensor flags
171  Flags readFlags();
172 
173  private:
174  jevois::CameraSensor itsSensor;
175  std::vector<std::shared_ptr<jevois::CameraDevice>> itsDev;
176  int itsDevIdx = -1, itsDev2Idx = -1;
177  int itsFd = -1, itsFd2 = -1;
178 
179  mutable std::timed_mutex itsMtx;
180  };
181 
182 } // namespace jevois
jevois::Camera::done
void done(RawImage &img) override
Indicate that user processing is done with an image previously obtained via get()
Definition: Camera.C:218
jevois::Camera::JEVOIS_SENSOR_ICM20948
@ JEVOIS_SENSOR_ICM20948
Definition: Camera.H:167
jevois::Camera::JEVOIS_SENSOR_MONO
@ JEVOIS_SENSOR_MONO
Definition: Camera.H:166
jevois::VideoInput
Base class for video input, which will get derived into Camera and MovieInput.
Definition: VideoInput.H:31
jevois::Camera::queryMenu
void queryMenu(struct v4l2_querymenu &qm) const override
Get the available menu entry names for a menu-type control, throw if unsupported by hardware.
Definition: Camera.C:244
jevois::RawImage
A raw image as coming from a V4L2 Camera and/or being sent out to a USB Gadget.
Definition: RawImage.H:110
jevois::Camera::done2
void done2(RawImage &img) override
Indicate that user processing is done with an image previously obtained via get2()
Definition: Camera.C:227
CameraDevice.H
VideoInput.H
jevois::Camera::~Camera
~Camera()
Close the device and free all resources.
Definition: Camera.C:154
jevois
Definition: Concepts.dox:1
jevois::Camera::readFlags
Flags readFlags()
Get the sensor flags.
Definition: Camera.C:313
jevois::Camera::queryControl
void queryControl(struct v4l2_queryctrl &qc) const override
Get information about a control, throw if unsupported by hardware.
Definition: Camera.C:236
jevois::Camera::get
void get(RawImage &img) override
Get the next captured buffer.
Definition: Camera.C:193
jevois::Camera::readRegister
unsigned short readRegister(unsigned short reg)
Read a value from one of the camera's registers.
Definition: Camera.C:287
VideoBuffers.H
jevois::Camera::streamOn
void streamOn() override
Start streaming.
Definition: Camera.C:166
jevois::Camera::lock
int lock()
Lock the camera and return its file descriptor.
Definition: Camera.C:299
jevois::Camera::getControl
void getControl(struct v4l2_control &ctrl) const
Get a control's current value, throw if unsupported by hardware.
Definition: Camera.C:252
jevois::Camera::setFormat
void setFormat(VideoMapping const &m) override
Set the video format and frame rate.
Definition: Camera.C:51
jevois::Camera::unlock
void unlock()
Unlock the camera that was previously locked by lock()
Definition: Camera.C:307
jevois::Camera::Flags
Flags
Sensor flags.
Definition: Camera.H:163
jevois::Camera::streamOff
void streamOff() override
Stop streaming.
Definition: Camera.C:184
jevois::Camera::Camera
Camera(std::string const &devname, jevois::CameraSensor s=jevois::CameraSensor::any, unsigned int const nbufs=0)
Construct and open the device.
Definition: Camera.C:33
jevois::Camera::hasScaledImage
bool hasScaledImage() const override
Check whether a second input image scaled by the JeVoisPro Platform ISP is available.
Definition: Camera.C:202
jevois::Camera::setControl
void setControl(struct v4l2_control const &ctrl) override
Set a control, throw if control not supported or the hardware rejects the value.
Definition: Camera.C:264
jevois::Camera::get2
void get2(RawImage &img) override
Get the next captured buffer, for second ISP-scaled image.
Definition: Camera.C:208
jevois::Camera
JeVois camera driver class - grabs frames from a Video4Linux camera sensor.
Definition: Camera.H:62
jevois::Camera::abortStream
void abortStream() override
Abort streaming.
Definition: Camera.C:175
jevois::Camera::JEVOIS_SENSOR_COLOR
@ JEVOIS_SENSOR_COLOR
Definition: Camera.H:165
jevois::Camera::writeRegister
void writeRegister(unsigned short reg, unsigned short val)
Write a value to one of the camera's registers.
Definition: Camera.C:276