JeVoisBase  1.17
JeVois Smart Embedded Machine Vision Toolkit Base Modules
Share this page:
ArUco.H
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #pragma once
19 
21 #include <jevois/Types/Enum.H>
22 #include <jevois/GPU/GUIhelper.H>
23 
24 #include <opencv2/aruco.hpp>
25 
26 namespace jevois { class StdModule; }
27 
28 namespace aruco
29 {
30  static jevois::ParameterCategory const ParamCateg("ArUco Options");
31 
32  //! Parameter \relates ArUco
33  JEVOIS_DECLARE_PARAMETER(camparams, std::string, "File stem of camera parameters, or empty. Camera resolution "
34  "will be appended, as well as a .yaml extension. For example, specifying 'calibration' "
35  "here and running the camera sensor at 320x240 will attempt to load "
36  "calibration320x240.yaml from within directory " JEVOIS_SHARE_PATH "/camera/ - Note that "
37  "this parameter cannot be changed at runtime (must be set in the module's params.cfg).",
38  "calibration", ParamCateg);
39 
40  //! Parameter \relates ArUco
41  JEVOIS_DECLARE_PARAMETER(detparams, std::string, "Filename of detector parameters, or empty - Note that "
42  "this parameter cannot be changed at runtime (must be set in the module's params.cfg).",
43  "", ParamCateg);
44 
45  //! Enum for parameter \relates ArUco
46  JEVOIS_DEFINE_ENUM_CLASS(Dict, (Original) (D4X4_50) (D4X4_100) (D4X4_250) (D4X4_1000) (D5X5_50) (D5X5_100)
47  (D5X5_250) (D5X5_1000) (D6X6_50) (D6X6_100) (D6X6_250) (D6X6_1000) (D7X7_50)
48  (D7X7_100) (D7X7_250) (D7X7_1000) (ATAG_16h5) (ATAG_25h9) (ATAG_36h10) (ATAG_36h11) );
49 
50  //! Parameter \relates ArUco
51  JEVOIS_DECLARE_PARAMETER(dictionary, Dict, "Symbol dictionary to use - Note that "
52  "this parameter cannot be changed at runtime (must be set in the module's params.cfg).",
53  Dict::D4X4_50, Dict_Values, ParamCateg);
54 
55  //! Parameter \relates ArUco
56  JEVOIS_DECLARE_PARAMETER(dopose, bool, "Compute (and show) pose vectors, requires a valid camera calibration",
57  false, ParamCateg);
58 
59  //! Parameter \relates ArUco
60  JEVOIS_DECLARE_PARAMETER(markerlen, float, "Marker side length (millimeters), used only for pose estimation",
61  100.0F, ParamCateg);
62 
63  //! Parameter \relates ArUco
64  JEVOIS_DECLARE_PARAMETER(showcube, bool, "Show a 3D cube on top of each detected marker, when dopose is also true",
65  false, aruco::ParamCateg);
66 }
67 
68 //! Simple wrapper class over the opencv_contrib ArUco augmented reality markers
69 /*! ArUco markers are small 2D barcodes. Each ArUco marker corresponds to a number, encoded into a small grid of black
70  and white pixels. The ArUco decoding algorithm is capable of locating, decoding, and of estimating the pose
71  (location and orientation in space) of any ArUco markers in the camera's field of view.
72 
73  ArUcos are very useful as tags for many robotics and augmented reality applications. For example, one may place an
74  ArUco next to a robot's charging station, an elevator button, or an object that a robot should manipulate.
75 
76  For more information about ArUco, see https://www.uco.es/investiga/grupos/ava/node/26
77 
78  The implementation of ArUco used by JeVois is the one of OpenCV-Contrib, documented here:
79  http://docs.opencv.org/3.2.0/d5/dae/tutorial_aruco_detection.html
80 
81  ArUco markers can be created with several standard dictionaries. Different disctionaries give rise to different
82  numbers of pixels in the markers, and to different numbers of possible symbols that can be created using the
83  dictionary. The default dictionary used by JeVois is 4x4 with 50 symbols. Other dictionaries are also supported by
84  setting the appropriate parameter over serial port or in a config file, up to 7x7 with 1000 symbols.
85 
86  Creating and printing ArUco markers
87  -----------------------------------
88 
89  To create some markers, have a look at the samples here:
90  https://github.com/opencv/opencv_contrib/tree/master/modules/aruco
91 
92  To make sure that the files you compile are compatible with JeVois which uses OpenCV 3.2.0 release, get that exact
93  release from GitHub as opposed to just the latest version of those files:
94  \verbatim
95  wget wget https://github.com/opencv/opencv_contrib/archive/3.2.0.tar.gz
96  tar zxvf 3.2.0.tar.gz
97  # The files referenced below are in: opencv_contrib-3.2.0/modules/aruco/samples/
98  \endverbatim
99 
100  To compile the sample program that can generate some ArUco markers (e.g., to be printed on paper):
101  \code
102  g++ -I/usr/share/jevois-opencv-3.2.0/include -L/usr/share/jevois-opencv-3.2.0/lib create_marker.cpp \\
103  -o create_marker -lopencv_core -lopencv_imgcodecs -lopencv_highgui -lopencv_aruco
104  \endcode
105 
106  Then, to make images of the markers in dictionary 0 (4x4_50):
107  \code
108  for id in {0..49}; do ./create_marker -d=0 --id=${id} aruco${id}.png; done
109  \endcode
110 
111  You can then print them and later detect them using this Component. Make sure you select the same dictionary in the
112  component as you did when you generated the markers.
113 
114  Recovering 3D pose of markers
115  -----------------------------
116 
117  To enable recovery of the 3D pose of a marker, you need to calibrate your camera. Again using the sample code:
118 
119  \code
120  g++ -I/usr/share/jevois-opencv-3.2.0/include -L/usr/share/jevois-opencv-3.2.0/lib create_board_charuco.cpp \\
121  -o create_board_charuco -lopencv_core -lopencv_imgcodecs -lopencv_highgui -lopencv_aruco -lopencv_imgproc -lopencv_videoio
122 
123  g++ -I/usr/share/jevois-opencv-3.2.0/include -L/usr/share/jevois-opencv-3.2.0/lib calibrate_camera_charuco.cpp \\
124  -o calibrate_camera_charuco -lopencv_core -lopencv_imgcodecs -lopencv_highgui -lopencv_aruco -lopencv_imgproc -lopencv_videoio
125  \endcode
126 
127  To create a ChArUco board that can be printed, using dictionary 0, and then derive the camera parameters from it
128  (set the `--ml` and `--sl` parameters to what you measure on your printed board; below are what we measured after
129  printing the 5x8 charuco board on US Letter paper with auto scaling/rotate to fit paper):
130 
131  \code
132  ./create_board_charuco -d=0 -h=5 -w=8 --ml=200 --sl=350 charuco.png
133  ./calibrate_camera_charuco -d=0 -h=5 -w=8 --ml=.0172 --sl=.0303 --rs --sc calibration.yaml
134  \endcode
135 
136  \ingroup components */
137 class ArUco : public jevois::Component,
138  public jevois::Parameter<aruco::camparams, aruco::detparams, aruco::dictionary,
139  aruco::dopose, aruco::markerlen, aruco::showcube>
140 {
141  public:
142  //! Constructor
144 
145  //! Destructor
146  virtual ~ArUco();
147 
148  //! Initialize, create the detector and read the config files
149  void postInit() override;
150 
151  //! Un-initialize, nuke allocated resources
152  void postUninit() override;
153 
154  //! Detect markers
155  void detectMarkers(cv::InputArray image, cv::OutputArray ids, cv::OutputArrayOfArrays corners);
156 
157  //! Estimate pose of individual markers
158  void estimatePoseSingleMarkers(cv::InputArrayOfArrays corners, cv::OutputArray rvecs, cv::OutputArray tvecs);
159 
160  //! Draw any markers previously detected by detectMarkers()
161  /*! If txtx,txty are positive, also print a text string there */
162  void drawDetections(jevois::RawImage & outimg, int txtx, int txty, std::vector<int> ids,
163  std::vector<std::vector<cv::Point2f> > corners, std::vector<cv::Vec3d> const & rvecs,
164  std::vector<cv::Vec3d> const & tvecs);
165 
166 #ifdef JEVOIS_PRO
167  //! Draw any markers previously detected by detectMarkers()
168  void drawDetections(jevois::GUIhelper & helper, std::vector<int> ids,
169  std::vector<std::vector<cv::Point2f> > corners, std::vector<cv::Vec3d> const & rvecs,
170  std::vector<cv::Vec3d> const & tvecs);
171 #endif
172 
173  //! Send serial messages about detections
174  /*! The module given should be the owner of this component, we will use it to actually send each serial message
175  using some variant of jevois::Module::sendSerial(). */
176  void sendSerial(jevois::StdModule * mod, std::vector<int> ids, std::vector<std::vector<cv::Point2f> > corners,
177  unsigned int w, unsigned int h, std::vector<cv::Vec3d> const & rvecs,
178  std::vector<cv::Vec3d> const & tvecs);
179 
180  //! Our current camera matrix
181  cv::Mat itsCamMatrix;
182 
183  //! Our current distortion coefficients
184  cv::Mat itsDistCoeffs;
185 
186  protected:
187  cv::Ptr<cv::aruco::DetectorParameters> itsDetectorParams;
188  cv::Ptr<cv::aruco::Dictionary> itsDictionary;
189 };
190 
ArUco::detectMarkers
void detectMarkers(cv::InputArray image, cv::OutputArray ids, cv::OutputArrayOfArrays corners)
Detect markers.
Definition: ArUco.C:137
jevois::ParameterRegistry::Component
friend friend class Component
jevois::GUIhelper
JEVOIS_DECLARE_PARAMETER
JEVOIS_DECLARE_PARAMETER(thresh1, double, "First threshold for hysteresis", 50.0, ParamCateg)
JEVOIS_DEFINE_ENUM_CLASS
JEVOIS_DEFINE_ENUM_CLASS(CameraSensor,(any)(ov9650)(ov2640)(ov7725)(ar0135)(imx290))
jevois::Component
ArUco::postUninit
void postUninit() override
Un-initialize, nuke allocated resources.
Definition: ArUco.C:125
jevois::RawImage
ArUco
Simple wrapper class over the opencv_contrib ArUco augmented reality markers.
Definition: ArUco.H:137
jevois::ParameterCategory
ArUco::estimatePoseSingleMarkers
void estimatePoseSingleMarkers(cv::InputArrayOfArrays corners, cv::OutputArray rvecs, cv::OutputArray tvecs)
Estimate pose of individual markers.
Definition: ArUco.C:163
jevois
F
float F
ArUco::~ArUco
virtual ~ArUco()
Destructor.
Definition: ArUco.C:27
Component.H
ArUco::itsDistCoeffs
cv::Mat itsDistCoeffs
Our current distortion coefficients.
Definition: ArUco.H:184
Enum.H
ArUco::drawDetections
void drawDetections(jevois::RawImage &outimg, int txtx, int txty, std::vector< int > ids, std::vector< std::vector< cv::Point2f > > corners, std::vector< cv::Vec3d > const &rvecs, std::vector< cv::Vec3d > const &tvecs)
Draw any markers previously detected by detectMarkers()
Definition: ArUco.C:206
ArUco::itsDictionary
cv::Ptr< cv::aruco::Dictionary > itsDictionary
Definition: ArUco.H:188
aruco
Definition: ArUco.H:28
h
int h
ArUco::itsCamMatrix
cv::Mat itsCamMatrix
Our current camera matrix.
Definition: ArUco.H:181
ArUco::postInit
void postInit() override
Initialize, create the detector and read the config files.
Definition: ArUco.C:31
jevois::StdModule
ArUco::sendSerial
void sendSerial(jevois::StdModule *mod, std::vector< int > ids, std::vector< std::vector< cv::Point2f > > corners, unsigned int w, unsigned int h, std::vector< cv::Vec3d > const &rvecs, std::vector< cv::Vec3d > const &tvecs)
Send serial messages about detections.
Definition: ArUco.C:169
GUIhelper.H
ArUco::itsDetectorParams
cv::Ptr< cv::aruco::DetectorParameters > itsDetectorParams
Definition: ArUco.H:187