JeVoisBase  1.18
JeVois Smart Embedded Machine Vision Toolkit Base Modules
Share this page:
DemoQRcode.C
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #include <jevois/Core/Module.H>
19 #include <jevois/Debug/Timer.H>
22 #include <opencv2/core/core.hpp>
23 #include <opencv2/imgproc/imgproc.hpp>
24 
25 // icon by Vaadin in shapes at flaticon
26 
27 //! Simple demo of QR-code and barcode detection and decoding using the ZBar library
28 /*! Detect barcodes and QR-codes, and decode their contents.
29 
30  QR-codes (Quick Response Codes) are popular 2D patterns that contain embedded information, such as a string of text,
31  a URL, etc. They basically work like barcodes, coding information into a high-contrast, geometric pattern that is
32  easier to detect and decode by a machine that more conventional human-written text or drawings.
33 
34  One can generate QR-codes containing different kinds of information, for example using online QR-code generators,
35  such as http://www.qr-code-generator.com/
36 
37  JeVois detects and decodes QR-codes and other barcodes. The implementation of the detection and decoding algorithm
38  used in JeVois is from the popular library ZBar, found at http://zbar.sourceforge.net/
39 
40  You should be able to sustain 30 frames/s with camera resolution 320x240, and 15 frames/s with camera resolution
41  640x480 when running this module inside the JeVois smart camera.
42 
43  Serial Messages
44  ---------------
45 
46  This module can send standardized serial messages as described in \ref UserSerialStyle. One message is issued for
47  every detected QR-code or barcode, on every video frame.
48 
49  - Serial message type: \b 2D
50  - `id`: type of symbol (e.g., \b QR-Code, \b ISBN13, etc).
51  - `x`, `y`, or vertices: standardized 2D coordinates of symbol center or of corners of bounding box
52  (depending on \p serstyle)
53  - `w`, `h`: standardized object size
54  - `extra`: decoded contents (e.g., URL that was in a QR-code, ISBN number from a barcode, etc)
55 
56  Note that when \p serstyle is \b Fine, only 4 corners are returned for each detected QR-code, but many points are
57  returned all over each detected barcode. Beware to not exceed your serial bandwidth in that case.
58 
59  See \ref UserSerialStyle for more on standardized serial messages, and \ref coordhelpers for more info on
60  standardized coordinates.
61 
62  @author Laurent Itti
63 
64  @displayname Demo QR-code
65  @videomapping YUYV 640 526 15.0 YUYV 640 480 15.0 JeVois DemoQRcode
66  @videomapping YUYV 320 286 30.0 YUYV 320 240 30.0 JeVois DemoQRcode
67  @videomapping NONE 0 0 0 YUYV 640 480 15.0 JeVois DemoQRcode
68  @videomapping NONE 0 0 0 YUYV 320 240 30.0 JeVois DemoQRcode
69  @email itti\@usc.edu
70  @address University of Southern California, HNB-07A, 3641 Watt Way, Los Angeles, CA 90089-2520, USA
71  @copyright Copyright (C) 2016 by Laurent Itti, iLab and the University of Southern California
72  @mainurl http://jevois.org
73  @supporturl http://jevois.org/doc
74  @otherurl http://iLab.usc.edu
75  @license GPL v3
76  @distribution Unrestricted
77  @restrictions None
78  \ingroup modules */
80 {
81  public:
82 
83  // ####################################################################################################
84  //! Constructor
85  // ####################################################################################################
86  DemoQRcode(std::string const & instance) : jevois::StdModule(instance)
87  { itsQRcode = addSubComponent<QRcode>("qrcode"); }
88 
89  // ####################################################################################################
90  //! Virtual destructor for safe inheritance
91  // ####################################################################################################
92  virtual ~DemoQRcode()
93  { }
94 
95  // ####################################################################################################
96  //! Processing function, no video output
97  // ####################################################################################################
98  virtual void process(jevois::InputFrame && inframe) override
99  {
100  // Wait for next available camera image:
101  jevois::RawImage const inimg = inframe.get();
102  unsigned int const w = inimg.width, h = inimg.height;
103 
104  // Convert the image to grayscale:
105  cv::Mat grayimg = jevois::rawimage::convertToCvGray(inimg);
106 
107  // Let camera know we are done processing the input image:
108  inframe.done();
109 
110  // Process gray image through zbar:
111  zbar::Image zgray(grayimg.cols, grayimg.rows, "Y800", grayimg.data, grayimg.total());
112  itsQRcode->process(zgray);
113 
114  // Send all the results over serial:
115  itsQRcode->sendSerial(this, zgray, w, h);
116 
117  // Cleanup zbar image data:
118  zgray.set_data(nullptr, 0);
119  }
120 
121  // ####################################################################################################
122  //! Processing function with video output to USB
123  // ####################################################################################################
124  virtual void process(jevois::InputFrame && inframe, jevois::OutputFrame && outframe) override
125  {
126  static jevois::Timer timer("processing", 60, LOG_DEBUG);
127  static unsigned short const txtcol = jevois::yuyv::White;
128  size_t const nshow = 4; // number of lines to show
129 
130  // Wait for next available camera image:
131  jevois::RawImage const inimg = inframe.get();
132 
133  timer.start();
134 
135  // We only handle one specific pixel format, any size in this demo:
136  unsigned int const w = inimg.width, h = inimg.height;
137  inimg.require("input", w, h, V4L2_PIX_FMT_YUYV);
138 
139  // While we process it, start a thread to wait for out frame and paste the input into it:
140  jevois::RawImage outimg;
141  auto paste_fut = jevois::async([&]() {
142  outimg = outframe.get();
143  outimg.require("output", w, h + nshow * 10 + 6, inimg.fmt);
144  jevois::rawimage::paste(inimg, outimg, 0, 0);
145  jevois::rawimage::writeText(outimg, "JeVois QR-code + Barcode Detection Demo", 3, 3, txtcol);
146  jevois::rawimage::drawFilledRect(outimg, 0, h, w, outimg.height-h, 0x8000);
147  });
148 
149  // Convert the image to grayscale and process it through zbar:
150  cv::Mat grayimg = jevois::rawimage::convertToCvGray(inimg);
151  zbar::Image zgray(grayimg.cols, grayimg.rows, "Y800", grayimg.data, grayimg.total());
152  itsQRcode->process(zgray);
153 
154  // Wait for paste to finish up:
155  paste_fut.get();
156 
157  // Let camera know we are done processing the input image:
158  inframe.done();
159 
160  // Draw all detections:
161  itsQRcode->drawDetections(outimg, 3, h + 3, zgray, w, h, nshow);
162 
163  // Send all serial messages:
164  itsQRcode->sendSerial(this, zgray, w, h);
165 
166  // Cleanup zbar image data:
167  zgray.set_data(nullptr, 0);
168 
169  // Show processing fps:
170  std::string const & fpscpu = timer.stop();
171  jevois::rawimage::writeText(outimg, fpscpu, 3, h - 13, txtcol);
172 
173  // Send the output image with our processing results to the host over USB:
174  outframe.send();
175  }
176 
177 #ifdef JEVOIS_PRO
178  // ####################################################################################################
179  //! Processing function with JeVois-Pro GUI
180  // ####################################################################################################
181  virtual void process(jevois::InputFrame && inframe, jevois::GUIhelper & helper) override
182  {
183  static jevois::Timer timer("processing", 60, LOG_DEBUG);
184 
185  // Start the GUI frame:
186  unsigned short winw, winh;
187  bool idle = helper.startFrame(winw, winh);
188 
189  // Draw the camera frame:
190  int x = 0, y = 0; unsigned short iw = 0, ih = 0;
191  helper.drawInputFrame("camera", inframe, x, y, iw, ih);
192 
193  // Wait for next available camera image:
194  jevois::RawImage const inimg = inframe.getp();
195  unsigned int const w = inimg.width, h = inimg.height;
196  helper.itext("JeVois-Pro QRcode/Barcode Detection");
197 
198  timer.start();
199 
200  // Convert the image to grayscale and process it through zbar:
201  cv::Mat grayimg = jevois::rawimage::convertToCvGray(inimg);
202  zbar::Image zgray(grayimg.cols, grayimg.rows, "Y800", grayimg.data, grayimg.total());
203  itsQRcode->process(zgray);
204 
205  // Draw all detections:
206  itsQRcode->drawDetections(helper, zgray, w, h);
207 
208  // Send all serial messages:
209  itsQRcode->sendSerial(this, zgray, w, h);
210 
211  // Cleanup zbar image data:
212  zgray.set_data(nullptr, 0);
213 
214  // Let camera know we are done processing the input image:
215  inframe.done();
216 
217  // Show processing fps:
218  std::string const & fpscpu = timer.stop();
219  helper.iinfo(inframe, fpscpu, winw, winh);
220 
221  // Render the image and GUI:
222  helper.endFrame();
223  }
224 #endif
225 
226  // ####################################################################################################
227  protected:
228  std::shared_ptr<QRcode> itsQRcode;
229 };
230 
231 // Allow the module to be loaded as a shared object (.so) file:
jevois::GUIhelper
jevois::OutputFrame
jevois::async
std::future< std::invoke_result_t< std::decay_t< Function >, std::decay_t< Args >... > > async(Function &&f, Args &&... args)
jevois::GUIhelper::startFrame
bool startFrame(unsigned short &w, unsigned short &h)
Timer.H
jevois::GUIhelper::itext
void itext(char const *txt, ImU32 const &col=IM_COL32_BLACK_TRANS, int line=-1)
Module.H
jevois::GUIhelper::endFrame
void endFrame()
DemoQRcode::process
virtual void process(jevois::InputFrame &&inframe) override
Processing function, no video output.
Definition: DemoQRcode.C:98
JEVOIS_REGISTER_MODULE
JEVOIS_REGISTER_MODULE(DemoQRcode)
jevois::RawImage
jevois::Timer::start
void start()
jevois::rawimage::convertToCvGray
cv::Mat convertToCvGray(RawImage const &src)
jevois::RawImage::require
void require(char const *info, unsigned int w, unsigned int h, unsigned int f) const
DemoQRcode::DemoQRcode
DemoQRcode(std::string const &instance)
Constructor.
Definition: DemoQRcode.C:86
jevois::RawImage::width
unsigned int width
DemoQRcode::~DemoQRcode
virtual ~DemoQRcode()
Virtual destructor for safe inheritance.
Definition: DemoQRcode.C:92
jevois::rawimage::writeText
void writeText(RawImage &img, std::string const &txt, int x, int y, unsigned int col, Font font=Font6x10)
jevois
DemoQRcode::itsQRcode
std::shared_ptr< QRcode > itsQRcode
Definition: DemoQRcode.C:228
jevois::Timer::stop
const std::string & stop(double *seconds)
jevois::rawimage::drawFilledRect
void drawFilledRect(RawImage &img, int x, int y, unsigned int w, unsigned int h, unsigned int col)
jevois::StdModule::StdModule
StdModule(std::string const &instance)
DemoQRcode
Simple demo of QR-code and barcode detection and decoding using the ZBar library.
Definition: DemoQRcode.C:79
RawImageOps.H
jevois::GUIhelper::drawInputFrame
void drawInputFrame(char const *name, InputFrame const &frame, int &x, int &y, unsigned short &w, unsigned short &h, bool noalias=false, bool casync=false)
jevois::RawImage::height
unsigned int height
jevois::GUIhelper::iinfo
void iinfo(jevois::InputFrame const &inframe, std::string const &fpscpu, unsigned short winw=0, unsigned short winh=0)
jevois::InputFrame
DemoQRcode::process
virtual void process(jevois::InputFrame &&inframe, jevois::OutputFrame &&outframe) override
Processing function with video output to USB.
Definition: DemoQRcode.C:124
jevois::rawimage::paste
void paste(RawImage const &src, RawImage &dest, int dx, int dy)
jevois::RawImage::fmt
unsigned int fmt
h
int h
jevois::StdModule
QRcode.H
demo.w
w
Definition: demo.py:85
jevois::Timer