JeVois  1.3
JeVois Smart Embedded Machine Vision Toolkit
Share this page:
Gadget.H
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #pragma once
19 
20 #include <mutex>
21 #include <future>
22 #include <deque>
23 #include <atomic>
24 #include <linux/usb/video.h> // for uvc_streaming_control
25 #include <linux/videodev2.h>
28 #include <jevois/Image/RawImage.H>
29 
30 // for UVC gadget specific definitions; yes, this is only in the kernel tree, kernel maintainers should expose those
31 // definitions in the standard headers instead:
32 //#include "../../../../lichee/linux-3.4/drivers/usb/gadget/uvc.h"
33 // We now include this file in our tree in so our code can compile without requiring the kernel source tree:
34 #include "uvc.h"
35 
36 namespace jevois
37 {
38  class VideoInput;
39  class Engine;
40  class VideoBuffers;
41 
42  //! JeVois gadget driver - exposes a uvcvideo interface to host computer connected over USB
43  /*! Gadget is a user-space interface to the Linux kernel's gadget driver implemented by JeVois. A USB gadget driver is
44  a USB driver that runs on a device, as opposed to a USB driver that runs on a host computer. Gadget drivers
45  implement USB functions such as USB printers, USB modems, or USB cameras. In JeVois, the Gadget driver makes the
46  JeVois hardware platform appear as a UVC (USB Video Class) camera. Because UVC has been standardized, no special
47  driver is needed on the host computer to start using the UVC device, as virtually all modern operating systems
48  already provide a built-in driver for UVC-compliant cameras.
49 
50  The internal operation of Gadget is quite complex, especially due to its tight interaction with the kernel-side
51  UVC gadget driver. Yet, for users, Gadget is just another VideoOutput device: One can set its format, obtain blank
52  image buffers from it using get(), paint results into those buffers, and send them to the host computer when
53  complete using send().
54 
55  Gadget implements a zero-copy, zero-wait access to output video frames, that is:
56  - the pixel data of the image you obtain via get() is directly the memory-mapped pixel buffer that the silicon
57  hardware on the JeVois chip will use via direct-memory-access (DMA) to stream the data out over the USB link;
58  - as soon as you call send() that buffer will be queued for sending over USB (as opposed to having a fixed,
59  regular interval at which images may be streamed out). Gadget has several image buffers, allowing one to be
60  streamed out over USB while another is being handed over for filling by your application via get(). These
61  buffers are recycled, i.e., once send() is called, the underlying buffer is streamed over USB and then sent back
62  to the Gadget for future access by your code.
63 
64  Most programmers will never use Gadget directly, instead using Engine and OutputFrame. \ingroup core */
65  class Gadget : public VideoOutput
66  {
67  public:
68  //! Construct and open the device
69  /*! A vaid non-null camera is required for this gadget to work. To avoid testing for a non-null camera on each
70  operation of the gadget, we only test once at construction and then assume the camera will remain operational
71  for the lifetime of the gadget. Use 0 for nbufs to set it automatically. */
72  Gadget(std::string const & devname, VideoInput * camera, Engine * engine, size_t const nbufs = 0);
73 
74  //! Close the device and free all resources
75  virtual ~Gadget();
76 
77  //! Set the video format and frame rate
78  void setFormat(jevois::VideoMapping const & m) override;
79 
80  //! Get a pre-allocated image so that we can fill the pixel data and later send out over USB using send()
81  /*! May throw if not buffer is available, i.e., all have been queued to send to the host but have not yet been
82  sent. Application code must balance exactly one send() for each get(). */
83  void get(RawImage & img) override;
84 
85  //! Send an image out over USB to the host computer
86  /*! May throw if the format is incorrect or std::overflow_error if we have not yet consumed the previous image. */
87  void send(RawImage const & img) override;
88 
89  //! Start streaming
90  void streamOn() override;
91 
92  //! Abort streaming
93  /*! This only cancels future get() and done() calls, one should still call streamOff() to turn off streaming. */
94  void abortStream() override;
95 
96  //! Stop streaming
97  void streamOff() override;
98 
99  private:
100  volatile int itsFd;
101  size_t itsNbufs;
102  VideoBuffers * itsBuffers;
103  VideoInput * itsCamera;
104  Engine * itsEngine;
105 
106  void run(); // Function to service requests, runs in a separate thread
107  std::future<void> itsRunFuture;
108  std::atomic<bool> itsRunning;
109 
110  void processEvents();
111  void processEventSetup(struct usb_ctrlrequest const & ctrl, struct uvc_request_data & resp);
112  void processEventStandard(struct usb_ctrlrequest const & ctrl, struct uvc_request_data & resp);
113  void processEventClass(struct usb_ctrlrequest const & ctrl, struct uvc_request_data & resp);
114  void processEventData(struct uvc_request_data & data);
115  void processEventControlData(struct uvc_request_data & data);
116  void processEventControl(uint8_t req, uint8_t cs, uint8_t entity_id, uint8_t len, struct uvc_request_data & resp);
117  void processEventStreaming(uint8_t req, uint8_t cs, struct uvc_request_data & resp);
118  void processVideo();
119 
120  struct v4l2_format itsFormat;
121  float itsFps;
122  void fillStreamingControl(struct uvc_streaming_control * ctrl, VideoMapping const & m);
123  std::atomic<bool> itsStreaming;
124  int itsErrorCode;
125  int itsControl;
126  int itsEntity;
127  struct uvc_streaming_control itsProbe;
128  struct uvc_streaming_control itsCommit;
129 
130  std::deque<RawImage> itsImageQueue;
131  std::deque<size_t> itsDoneImgs;
132 
133  mutable std::timed_mutex itsMtx;
134  };
135 
136 } // namespace jevois
137 
void abortStream() override
Abort streaming.
Definition: Gadget.C:760
void send(RawImage const &img) override
Send an image out over USB to the host computer.
Definition: Gadget.C:839
virtual ~Gadget()
Close the device and free all resources.
Definition: Gadget.C:180
void streamOn() override
Start streaming.
Definition: Gadget.C:706
Gadget(std::string const &devname, VideoInput *camera, Engine *engine, size_t const nbufs=0)
Construct and open the device.
Definition: Gadget.C:134
A raw image as coming from a V4L2 Camera and/or being sent out to a USB Gadget.
Definition: RawImage.H:110
Collection of buffers for V4L2 video frames (Camera or Gadget) with hooks to the MMAP'd areas...
Definition: VideoBuffers.H:41
Simple struct to hold video mapping definitions for the processing Engine.
Definition: VideoMapping.H:41
Base class for video input, which will get derived into Camera and MovieInput.
Definition: VideoInput.H:31
JeVois gadget driver - exposes a uvcvideo interface to host computer connected over USB...
Definition: Gadget.H:65
JeVois processing engine - gets images from camera sensor, processes them, and sends results over USB...
Definition: Engine.H:229
Base class for video output. Gadget, MovieOutput, VideoDisplay, and VideoOutputNone derive from it...
Definition: VideoOutput.H:27
void setFormat(jevois::VideoMapping const &m) override
Set the video format and frame rate.
Definition: Gadget.C:196
void streamOff() override
Stop streaming.
Definition: Gadget.C:768