JeVoisBase  1.5
JeVois Smart Embedded Machine Vision Toolkit Base Modules
Share this page:
ArUco.H
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #pragma once
19 
21 #include <jevois/Types/Enum.H>
22 #include <opencv2/aruco.hpp>
23 namespace jevois { class StdModule; }
24 
25 namespace aruco
26 {
27  static jevois::ParameterCategory const ParamCateg("ArUco Options");
28 
29  //! Parameter \relates ArUco
30  JEVOIS_DECLARE_PARAMETER(camparams, std::string, "File stem of camera parameters, or empty. Camera resolution "
31  "will be appended, as well as a .yaml extension. For example, specifying 'calibration' "
32  "here and running the camera sensor at 320x240 will attempt to load "
33  "calibration320x240.yaml from within the module's directory.",
34  "calibration", ParamCateg);
35 
36  //! Parameter \relates ArUco
37  JEVOIS_DECLARE_PARAMETER(detparams, std::string, "Filename of detector parameters, or empty", "", ParamCateg);
38 
39  //! Enum for parameter \relates ArUco
40  JEVOIS_DEFINE_ENUM_CLASS(Dict, (Original) (D4X4_50) (D4X4_100) (D4X4_250) (D4X4_1000) (D5X5_50) (D5X5_100)
41  (D5X5_250) (D5X5_1000) (D6X6_50) (D6X6_100) (D6X6_250) (D6X6_1000) (D7X7_50)
42  (D7X7_100) (D7X7_250) (D7X7_1000) );
43 
44  //! Parameter \relates ArUco
45  JEVOIS_DECLARE_PARAMETER(dictionary, Dict, "Symbol dictionary to use", Dict::D4X4_50, ParamCateg);
46 
47  //! Parameter \relates ArUco
48  JEVOIS_DECLARE_PARAMETER(dopose, bool, "Compute (and show) pose vectors, requires a valid camera calibration",
49  false, ParamCateg);
50 
51  //! Parameter \relates ArUco
52  JEVOIS_DECLARE_PARAMETER(markerlen, float, "Marker side length (millimeters), used only for pose estimation",
53  100.0F, ParamCateg);
54 
55  //! Parameter \relates ArUco
56  JEVOIS_DECLARE_PARAMETER(showcube, bool, "Show a 3D cube on top of each detected marker, when dopose is also true",
57  false, aruco::ParamCateg);
58 }
59 
60 //! Simple wrapper class over the opencv_contrib ArUco augmented reality markers
61 /*! ArUco markers are small 2D barcodes. Each ArUco marker corresponds to a number, encoded into a small grid of black
62  and white pixels. The ArUco decoding algorithm is capable of locating, decoding, and of estimating the pose
63  (location and orientation in space) of any ArUco markers in the camera's field of view.
64 
65  ArUcos are very useful as tags for many robotics and augmented reality applications. For example, one may place an
66  ArUco next to a robot's charging station, an elevator button, or an object that a robot should manipulate.
67 
68  For more information about ArUco, see https://www.uco.es/investiga/grupos/ava/node/26
69 
70  The implementation of ArUco used by JeVois is the one of OpenCV-Contrib, documented here:
71  http://docs.opencv.org/3.2.0/d5/dae/tutorial_aruco_detection.html
72 
73  ArUco markers can be created with several standard dictionaries. Different disctionaries give rise to different
74  numbers of pixels in the markers, and to different numbers of possible symbols that can be created using the
75  dictionary. The default dictionary used by JeVois is 4x4 with 50 symbols. Other dictionaries are also supported by
76  setting the appropriate parameter over serial port or in a config file, up to 7x7 with 1000 symbols.
77 
78  Creating and printing ArUco markers
79  -----------------------------------
80 
81  To create some markers, have a look at the samples here:
82  https://github.com/opencv/opencv_contrib/tree/master/modules/aruco
83 
84  To make sure that the files you compile are compatible with JeVois which uses OpenCV 3.2.0 release, get that exact
85  release from GitHub as opposed to just the latest version of those files:
86  \verbatim
87  wget wget https://github.com/opencv/opencv_contrib/archive/3.2.0.tar.gz
88  tar zxvf 3.2.0.tar.gz
89  # The files referenced below are in: opencv_contrib-3.2.0/modules/aruco/samples/
90  \endverbatim
91 
92  To compile the sample program that can generate some ArUco markers (e.g., to be printed on paper):
93  \code
94  g++ -I/usr/share/jevois-opencv-3.2.0/include -L/usr/share/jevois-opencv-3.2.0/lib create_marker.cpp \\
95  -o create_marker -lopencv_core -lopencv_imgcodecs -lopencv_highgui -lopencv_aruco
96  \endcode
97 
98  Then, to make images of the markers in dictionary 0 (4x4_50):
99  \code
100  for id in {0..49}; do ./create_marker -d=0 --id=${id} aruco${id}.png; done
101  \endcode
102 
103  You can then print them and later detect them using this Component. Make sure you select the same dictionary in the
104  component as you did when you generated the markers.
105 
106  Recovering 3D pose of markers
107  -----------------------------
108 
109  To enable recovery of the 3D pose of a marker, you need to calibrate your camera. Again using the sample code:
110 
111  \code
112  g++ -I/usr/share/jevois-opencv-3.2.0/include -L/usr/share/jevois-opencv-3.2.0/lib create_board_charuco.cpp \\
113  -o create_board_charuco -lopencv_core -lopencv_imgcodecs -lopencv_highgui -lopencv_aruco -lopencv_imgproc -lopencv_videoio
114 
115  g++ -I/usr/share/jevois-opencv-3.2.0/include -L/usr/share/jevois-opencv-3.2.0/lib calibrate_camera_charuco.cpp \\
116  -o calibrate_camera_charuco -lopencv_core -lopencv_imgcodecs -lopencv_highgui -lopencv_aruco -lopencv_imgproc -lopencv_videoio
117  \endcode
118 
119  To create a ChArUco board that can be printed, using dictionary 0, and then derive the camera parameters from it
120  (set the `--ml` and `--sl` parameters to what you measure on your printed board; below are what we measured after
121  printing the 5x8 charuco board on US Letter paper with auto scaling/rotate to fit paper):
122 
123  \code
124  ./create_board_charuco -d=0 -h=5 -w=8 --ml=200 --sl=350 charuco.png
125  ./calibrate_camera_charuco -d=0 -h=5 -w=8 --ml=.0172 --sl=.0303 --rs --sc calibration.yaml
126  \endcode
127 
128  \ingroup components */
129 class ArUco : public jevois::Component,
130  public jevois::Parameter<aruco::camparams, aruco::detparams, aruco::dictionary,
131  aruco::dopose, aruco::markerlen, aruco::showcube>
132 {
133  public:
134  //! Constructor
136 
137  //! Destructor
138  virtual ~ArUco();
139 
140  //! Initialize, create the detector and read the config files
141  void postInit() override;
142 
143  //! Un-initialize, nuke allocated resources
144  void postUninit() override;
145 
146  //! Detect markers
147  void detectMarkers(cv::InputArray image, cv::OutputArray ids, cv::OutputArrayOfArrays corners);
148 
149  //! Estimate pose of individual markers
150  void estimatePoseSingleMarkers(cv::InputArrayOfArrays corners, cv::OutputArray rvecs, cv::OutputArray tvecs);
151 
152  //! Draw any markers previously detected by detectMarkers()
153  /*! If txtx,txty are positive, also print a text string there */
154  void drawDetections(jevois::RawImage & outimg, int txtx, int txty, std::vector<int> ids,
155  std::vector<std::vector<cv::Point2f> > corners, std::vector<cv::Vec3d> const & rvecs,
156  std::vector<cv::Vec3d> const & tvecs);
157 
158  //! Send serial messages about detections
159  /*! The module given should be the owner of this component, we will use it to actually send each serial message
160  using some variant of jevois::Module::sendSerial(). */
161  void sendSerial(jevois::StdModule * mod, std::vector<int> ids, std::vector<std::vector<cv::Point2f> > corners,
162  unsigned int w, unsigned int h, std::vector<cv::Vec3d> const & rvecs,
163  std::vector<cv::Vec3d> const & tvecs);
164 
165  //! Our current camera matrix
166  cv::Mat itsCamMatrix;
167 
168  //! Our current distortion coefficients
169  cv::Mat itsDistCoeffs;
170 
171  protected:
172  cv::Ptr<cv::aruco::DetectorParameters> itsDetectorParams;
173  cv::Ptr<cv::aruco::Dictionary> itsDictionary;
174 };
175 
JEVOIS_DECLARE_PARAMETER(thresh1, double, "First threshold for hysteresis", 50.0, ParamCateg)
Simple wrapper class over the opencv_contrib ArUco augmented reality markers.
Definition: ArUco.H:129
Definition: ArUco.H:25
cv::Mat itsDistCoeffs
Our current distortion coefficients.
Definition: ArUco.H:169
cv::Mat itsCamMatrix
Our current camera matrix.
Definition: ArUco.H:166
friend friend class Component
cv::Ptr< cv::aruco::Dictionary > itsDictionary
Definition: ArUco.H:173
cv::Ptr< cv::aruco::DetectorParameters > itsDetectorParams
Definition: ArUco.H:172