JeVois  1.3
JeVois Smart Embedded Machine Vision Toolkit
Share this page:
Camera.H
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #pragma once
19 
20 #include <jevois/Core/VideoInput.H>
22 
23 #include <linux/videodev2.h>
24 #include <mutex>
25 #include <future>
26 #include <atomic>
27 
28 namespace jevois
29 {
30  //! JeVois camera driver class - grabs frames from a Video4Linux camera sensor
31  /*! On the platform hardware, the Camera class provides access to the camera sensor on the parallel camera bus
32  (CSI). On other hardware, it can provide access to any Video4Linux2 camera (e.g., USB webcam), although with some
33  limitations, such as:
34  - cannot read/write raw camera registers.
35  - usually cannot select any frames/s but instead limited to a few fixed values like 15fps, 30fps.
36  - usually limited to 3fps max (no 60fps or 120fps modes like on JeVois platform hardware)
37  - usually limited number of controls (exposure, gain, etc) available.
38 
39  The camera runs a thread that handles capturing frames automatically. Users can access the next grabbed frame
40  using get(), which may block until the frame is fully captured. Once they are finished with a frame obtained
41  through get(), users should hand it back to the Camera by calling done(). When using the JeVois Engine and a
42  Module, get() and done() are automatically called in an exception-safe manner through the use of the InputFrame
43  class. Indeed, video grabbing uses a fixed set of image buffers that are allocated by the Linux kernel when a
44  video format is configured, and the camera hardware then cycles through the buffers. On the platform, the camera
45  image buffers are allocated in the ION carveout memory area, which is a reserved memory zone configured in the
46  Linux kernel, and the USB output buffers are also allocated in that zone. To avoid reserving too much memory that
47  will not be used, the ION carveout reservation is quite small by default, but is sufficient for 3 1280x1024 YUYV
48  buffers.
49 
50  Camera implements a zero-copy, zero-wait access to input video frames, that is:
51  - the pixel data of the image you obtain via get() is directly the memory-mapped pixel buffer that the silicon
52  hardware on the JeVois chip uses via direct-memory-access (DMA) to stream the pixel data from the camera chip to
53  processor memory;
54  - as soon as an image is captured by the camera hardware, get() unblocks and returns it (as opposed to having a
55  fixed, regular interval at which images may be available). Camera has several image buffers, allowing one to be
56  captured while another is being handed over for processing via get(). These buffers are recycled, i.e., once
57  done() is called, the underlying buffer is sent back to the camera hardware for future capture.
58 
59  Camera will drop frames if they are not processed by user code fast enough. That is, if all buffers have been
60  grabbed already and not yet handed over through get() and then returned by done(), then the next camera sensor
61  frame will be dropped.
62 
63  Most programmers will never use Camera directly, instead using Engine and InputFrame. \ingroup core */
64  class Camera : public VideoInput
65  {
66  public:
67  //! Construct and open the device
68  /*! \param devname device name, e.g., /dev/video0
69  \param nbufs number of video grab buffers, or 0 for automatic. */
70  Camera(std::string const & devname, unsigned int const nbufs = 0);
71 
72  //! Close the device and free all resources
73  ~Camera();
74 
75  //! Start streaming
76  void streamOn() override;
77 
78  //! Abort streaming
79  /*! This only cancels future get() and done() calls, one should still call streamOff() to turn off streaming. */
80  void abortStream() override;
81 
82  //! Stop streaming
83  void streamOff() override;
84 
85  //! Get the next captured buffer
86  /*! Throws if we are not streaming or blocks until an image is available (has been captured). img should have been
87  created by the caller (with no pixel buffer allocated) and will be filled in by what we receive from the
88  device here. */
89  void get(RawImage & img) override;
90 
91  //! Indicate that user processing is done with an image previously obtained via get()
92  /*! You should call this as soon after get() as possible, once you are finished with the RawImage data so that it
93  can be recycled.
94 
95  Calling done() on a RawImage invalidates the image and in particular its pixel buffer. Users should make sure
96  that no attempt to use the image or the pixel buffer will be made after done() is called. */
97  void done(RawImage & img) override;
98 
99  //! Get information about a control, throw if unsupported by hardware
100  /*! Caller should zero-out qc and then set the id field to the desired control id. See VIDIOC_QUERYCTRL for more
101  information. */
102  void queryControl(struct v4l2_queryctrl & qc) const override;
103 
104  //! Get the available menu entry names for a menu-type control, throw if unsupported by hardware
105  /*! Caller should zero-out qm and then set the id and index fields to the desired control id and menu item
106  index. See VIDIOC_QUERYMENU for more information. */
107  void queryMenu(struct v4l2_querymenu & qm) const override;
108 
109  //! Get a control's current value, throw if unsupported by hardware
110  /*! This is just a pass-through to VIDIOC_G_CTRL, users should zero-out ctrl and then set in ctrl.id the desired
111  control ID. */
112  void getControl(struct v4l2_control & ctrl) const;
113 
114  //! Set a control, throw if control not supported or the hardware rejects the value
115  /*! This is just a pass-through to VIDIOC_S_CTRL */
116  void setControl(struct v4l2_control const & ctrl) override;
117 
118  //! Set the video format and frame rate
119  void setFormat(VideoMapping const & m) override;
120 
121  //! Write a value to one of the camera's registers
122  /*! This very low-level access is for development of optimal camera settings only and should not be used in normal
123  operation, it can crash your system. */
124  void writeRegister(unsigned char reg, unsigned char val) override;
125 
126  //! Read a value from one of the camera's registers
127  /*! This very low-level access is for development of optimal camera settings only and should not be used in normal
128  operation, it can crash your system. */
129  unsigned char readRegister(unsigned char reg) override;
130 
131  private:
132  int itsFd;
133  VideoBuffers * itsBuffers;
134  struct v4l2_format itsFormat;
135  std::atomic<bool> itsStreaming;
136  float itsFps;
137 
138  mutable std::condition_variable itsOutputCondVar;
139  mutable std::mutex itsOutputMtx;
140  RawImage itsOutputImage;
141  std::vector<size_t> itsDoneIdx;
142 
143  void run();
144  std::future<void> itsRunFuture;
145  std::atomic<bool> itsRunning;
146 
147  mutable std::timed_mutex itsMtx;
148  };
149 
150 } // namespace jevois
~Camera()
Close the device and free all resources.
Definition: Camera.C:179
void queryControl(struct v4l2_queryctrl &qc) const override
Get information about a control, throw if unsupported by hardware.
Definition: Camera.C:427
void writeRegister(unsigned char reg, unsigned char val) override
Write a value to one of the camera's registers.
Definition: Camera.C:459
void streamOff() override
Stop streaming.
Definition: Camera.C:355
unsigned char readRegister(unsigned char reg) override
Read a value from one of the camera's registers.
Definition: Camera.C:468
void setFormat(VideoMapping const &m) override
Set the video format and frame rate.
Definition: Camera.C:111
void streamOn() override
Start streaming.
Definition: Camera.C:298
void abortStream() override
Abort streaming.
Definition: Camera.C:342
void setControl(struct v4l2_control const &ctrl) override
Set a control, throw if control not supported or the hardware rejects the value.
Definition: Camera.C:449
void done(RawImage &img) override
Indicate that user processing is done with an image previously obtained via get() ...
Definition: Camera.C:411
A raw image as coming from a V4L2 Camera and/or being sent out to a USB Gadget.
Definition: RawImage.H:110
Collection of buffers for V4L2 video frames (Camera or Gadget) with hooks to the MMAP'd areas...
Definition: VideoBuffers.H:41
Camera(std::string const &devname, unsigned int const nbufs=0)
Construct and open the device.
Definition: Camera.C:56
JeVois camera driver class - grabs frames from a Video4Linux camera sensor.
Definition: Camera.H:64
Simple struct to hold video mapping definitions for the processing Engine.
Definition: VideoMapping.H:41
void queryMenu(struct v4l2_querymenu &qm) const override
Get the available menu entry names for a menu-type control, throw if unsupported by hardware...
Definition: Camera.C:433
Base class for video input, which will get derived into Camera and MovieInput.
Definition: VideoInput.H:31
void getControl(struct v4l2_control &ctrl) const
Get a control's current value, throw if unsupported by hardware.
Definition: Camera.C:439