JeVois  1.8
JeVois Smart Embedded Machine Vision Toolkit
Share this page:
Gadget.H
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #pragma once
19 
20 #include <mutex>
21 #include <future>
22 #include <deque>
23 #include <atomic>
24 #include <linux/usb/video.h> // for uvc_streaming_control
25 #include <linux/videodev2.h>
28 #include <jevois/Image/RawImage.H>
29 
30 // for UVC gadget specific definitions; yes, this is only in the kernel tree, kernel maintainers should expose those
31 // definitions in the standard headers instead:
32 //#include "../../../../lichee/linux-3.4/drivers/usb/gadget/uvc.h"
33 // We now include this file in our tree in so our code can compile without requiring the kernel source tree:
34 #include "uvc.h"
35 
36 namespace jevois
37 {
38  class VideoInput;
39  class Engine;
40  class VideoBuffers;
41 
42  //! JeVois gadget driver - exposes a uvcvideo interface to host computer connected over USB
43  /*! Gadget is a user-space interface to the Linux kernel's gadget driver implemented by JeVois. A USB gadget driver is
44  a USB driver that runs on a device, as opposed to a USB driver that runs on a host computer. Gadget drivers
45  implement USB functions such as USB printers, USB modems, or USB cameras. In JeVois, the Gadget driver makes the
46  JeVois hardware platform appear as a UVC (USB Video Class) camera. Because UVC has been standardized, no special
47  driver is needed on the host computer to start using the UVC device, as virtually all modern operating systems
48  already provide a built-in driver for UVC-compliant cameras.
49 
50  The internal operation of Gadget is quite complex, especially due to its tight interaction with the kernel-side
51  UVC gadget driver. Yet, for users, Gadget is just another VideoOutput device: One can set its format, obtain blank
52  image buffers from it using get(), paint results into those buffers, and send them to the host computer when
53  complete using send().
54 
55  Gadget implements a zero-copy, zero-wait access to output video frames, that is:
56  - the pixel data of the image you obtain via get() is directly the memory-mapped pixel buffer that the silicon
57  hardware on the JeVois chip will use via direct-memory-access (DMA) to stream the data out over the USB link;
58  - as soon as you call send() that buffer will be queued for sending over USB (as opposed to having a fixed,
59  regular interval at which images may be streamed out). Gadget has several image buffers, allowing one to be
60  streamed out over USB while another is being handed over for filling by your application via get(). These
61  buffers are recycled, i.e., once send() is called, the underlying buffer is streamed over USB and then sent back
62  to the Gadget for future access by your code.
63 
64  Most programmers will never use Gadget directly, instead using Engine and OutputFrame. \ingroup core */
65  class Gadget : public VideoOutput
66  {
67  public:
68  //! Construct and open the device
69  /*! A vaid non-null camera is required for this gadget to work. To avoid testing for a non-null camera on each
70  operation of the gadget, we only test once at construction and then assume the camera will remain operational
71  for the lifetime of the gadget. Use 0 for nbufs to set it automatically. */
72  Gadget(std::string const & devname, VideoInput * camera, Engine * engine, size_t const nbufs = 0,
73  bool multicam = false);
74 
75  //! Close the device and free all resources
76  virtual ~Gadget();
77 
78  //! Set the video format and frame rate
79  void setFormat(jevois::VideoMapping const & m) override;
80 
81  //! Get a pre-allocated image so that we can fill the pixel data and later send out over USB using send()
82  /*! May throw if not buffer is available, i.e., all have been queued to send to the host but have not yet been
83  sent. Application code must balance exactly one send() for each get(). */
84  void get(RawImage & img) override;
85 
86  //! Send an image out over USB to the host computer
87  /*! May throw if the format is incorrect or std::overflow_error if we have not yet consumed the previous image. */
88  void send(RawImage const & img) override;
89 
90  //! Start streaming
91  void streamOn() override;
92 
93  //! Abort streaming
94  /*! This only cancels future get() and done() calls, one should still call streamOff() to turn off streaming. */
95  void abortStream() override;
96 
97  //! Stop streaming
98  void streamOff() override;
99 
100  private:
101  volatile int itsFd;
102  bool const itsMulticam;
103  size_t itsNbufs;
104  VideoBuffers * itsBuffers;
105  VideoInput * itsCamera;
106  Engine * itsEngine;
107 
108  void run(); // Function to service requests, runs in a separate thread
109  std::future<void> itsRunFuture;
110  std::atomic<bool> itsRunning;
111 
112  void processEvents();
113  void processEventSetup(struct usb_ctrlrequest const & ctrl, struct uvc_request_data & resp);
114  void processEventStandard(struct usb_ctrlrequest const & ctrl, struct uvc_request_data & resp);
115  void processEventClass(struct usb_ctrlrequest const & ctrl, struct uvc_request_data & resp);
116  void processEventData(struct uvc_request_data & data);
117  void processEventControlData(struct uvc_request_data & data);
118  void processEventControl(uint8_t req, uint8_t cs, uint8_t entity_id, uint8_t len, struct uvc_request_data & resp);
119  void processEventStreaming(uint8_t req, uint8_t cs, struct uvc_request_data & resp);
120  void processVideo();
121 
122  struct v4l2_format itsFormat;
123  float itsFps;
124  void fillStreamingControl(struct uvc_streaming_control * ctrl, VideoMapping const & m);
125  std::atomic<bool> itsStreaming;
126  int itsErrorCode;
127  int itsControl;
128  int itsEntity;
129  struct uvc_streaming_control itsProbe;
130  struct uvc_streaming_control itsCommit;
131 
132  std::deque<RawImage> itsImageQueue;
133  std::deque<size_t> itsDoneImgs;
134 
135  mutable std::timed_mutex itsMtx;
136  };
137 
138 } // namespace jevois
139 
void abortStream() override
Abort streaming.
Definition: Gadget.C:773
void send(RawImage const &img) override
Send an image out over USB to the host computer.
Definition: Gadget.C:852
virtual ~Gadget()
Close the device and free all resources.
Definition: Gadget.C:184
void streamOn() override
Start streaming.
Definition: Gadget.C:718
A raw image as coming from a V4L2 Camera and/or being sent out to a USB Gadget.
Definition: RawImage.H:110
Collection of buffers for V4L2 video frames (Camera or Gadget) with hooks to the MMAP&#39;d areas...
Definition: VideoBuffers.H:41
Gadget(std::string const &devname, VideoInput *camera, Engine *engine, size_t const nbufs=0, bool multicam=false)
Construct and open the device.
Definition: Gadget.C:138
Simple struct to hold video mapping definitions for the processing Engine.
Definition: VideoMapping.H:41
Base class for video input, which will get derived into Camera and MovieInput.
Definition: VideoInput.H:31
JeVois gadget driver - exposes a uvcvideo interface to host computer connected over USB...
Definition: Gadget.H:65
JeVois processing engine - gets images from camera sensor, processes them, and sends results over USB...
Definition: Engine.H:257
Base class for video output. Gadget, MovieOutput, VideoDisplay, and VideoOutputNone derive from it...
Definition: VideoOutput.H:27
void setFormat(jevois::VideoMapping const &m) override
Set the video format and frame rate.
Definition: Gadget.C:200
void streamOff() override
Stop streaming.
Definition: Gadget.C:781