JeVoisBase  1.11
JeVois Smart Embedded Machine Vision Toolkit Base Modules
Share this page:
ArUco.H
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #pragma once
19 
21 #include <jevois/Types/Enum.H>
22 #include <opencv2/aruco.hpp>
23 namespace jevois { class StdModule; }
24 
25 namespace aruco
26 {
27  static jevois::ParameterCategory const ParamCateg("ArUco Options");
28 
29  //! Parameter \relates ArUco
30  JEVOIS_DECLARE_PARAMETER(camparams, std::string, "File stem of camera parameters, or empty. Camera resolution "
31  "will be appended, as well as a .yaml extension. For example, specifying 'calibration' "
32  "here and running the camera sensor at 320x240 will attempt to load "
33  "calibration320x240.yaml from within directory " JEVOIS_SHARE_PATH "/camera/ - Note that "
34  "this parameter cannot be changed at runtime (must be set in the module's params.cfg).",
35  "calibration", ParamCateg);
36 
37  //! Parameter \relates ArUco
38  JEVOIS_DECLARE_PARAMETER(detparams, std::string, "Filename of detector parameters, or empty - Note that "
39  "this parameter cannot be changed at runtime (must be set in the module's params.cfg).",
40  "", ParamCateg);
41 
42  //! Enum for parameter \relates ArUco
43  JEVOIS_DEFINE_ENUM_CLASS(Dict, (Original) (D4X4_50) (D4X4_100) (D4X4_250) (D4X4_1000) (D5X5_50) (D5X5_100)
44  (D5X5_250) (D5X5_1000) (D6X6_50) (D6X6_100) (D6X6_250) (D6X6_1000) (D7X7_50)
45  (D7X7_100) (D7X7_250) (D7X7_1000) );
46 
47  //! Parameter \relates ArUco
48  JEVOIS_DECLARE_PARAMETER(dictionary, Dict, "Symbol dictionary to use - Note that "
49  "this parameter cannot be changed at runtime (must be set in the module's params.cfg).",
50  Dict::D4X4_50, Dict_Values, ParamCateg);
51 
52  //! Parameter \relates ArUco
53  JEVOIS_DECLARE_PARAMETER(dopose, bool, "Compute (and show) pose vectors, requires a valid camera calibration",
54  false, ParamCateg);
55 
56  //! Parameter \relates ArUco
57  JEVOIS_DECLARE_PARAMETER(markerlen, float, "Marker side length (millimeters), used only for pose estimation",
58  100.0F, ParamCateg);
59 
60  //! Parameter \relates ArUco
61  JEVOIS_DECLARE_PARAMETER(showcube, bool, "Show a 3D cube on top of each detected marker, when dopose is also true",
62  false, aruco::ParamCateg);
63 }
64 
65 //! Simple wrapper class over the opencv_contrib ArUco augmented reality markers
66 /*! ArUco markers are small 2D barcodes. Each ArUco marker corresponds to a number, encoded into a small grid of black
67  and white pixels. The ArUco decoding algorithm is capable of locating, decoding, and of estimating the pose
68  (location and orientation in space) of any ArUco markers in the camera's field of view.
69 
70  ArUcos are very useful as tags for many robotics and augmented reality applications. For example, one may place an
71  ArUco next to a robot's charging station, an elevator button, or an object that a robot should manipulate.
72 
73  For more information about ArUco, see https://www.uco.es/investiga/grupos/ava/node/26
74 
75  The implementation of ArUco used by JeVois is the one of OpenCV-Contrib, documented here:
76  http://docs.opencv.org/3.2.0/d5/dae/tutorial_aruco_detection.html
77 
78  ArUco markers can be created with several standard dictionaries. Different disctionaries give rise to different
79  numbers of pixels in the markers, and to different numbers of possible symbols that can be created using the
80  dictionary. The default dictionary used by JeVois is 4x4 with 50 symbols. Other dictionaries are also supported by
81  setting the appropriate parameter over serial port or in a config file, up to 7x7 with 1000 symbols.
82 
83  Creating and printing ArUco markers
84  -----------------------------------
85 
86  To create some markers, have a look at the samples here:
87  https://github.com/opencv/opencv_contrib/tree/master/modules/aruco
88 
89  To make sure that the files you compile are compatible with JeVois which uses OpenCV 3.2.0 release, get that exact
90  release from GitHub as opposed to just the latest version of those files:
91  \verbatim
92  wget wget https://github.com/opencv/opencv_contrib/archive/3.2.0.tar.gz
93  tar zxvf 3.2.0.tar.gz
94  # The files referenced below are in: opencv_contrib-3.2.0/modules/aruco/samples/
95  \endverbatim
96 
97  To compile the sample program that can generate some ArUco markers (e.g., to be printed on paper):
98  \code
99  g++ -I/usr/share/jevois-opencv-3.2.0/include -L/usr/share/jevois-opencv-3.2.0/lib create_marker.cpp \\
100  -o create_marker -lopencv_core -lopencv_imgcodecs -lopencv_highgui -lopencv_aruco
101  \endcode
102 
103  Then, to make images of the markers in dictionary 0 (4x4_50):
104  \code
105  for id in {0..49}; do ./create_marker -d=0 --id=${id} aruco${id}.png; done
106  \endcode
107 
108  You can then print them and later detect them using this Component. Make sure you select the same dictionary in the
109  component as you did when you generated the markers.
110 
111  Recovering 3D pose of markers
112  -----------------------------
113 
114  To enable recovery of the 3D pose of a marker, you need to calibrate your camera. Again using the sample code:
115 
116  \code
117  g++ -I/usr/share/jevois-opencv-3.2.0/include -L/usr/share/jevois-opencv-3.2.0/lib create_board_charuco.cpp \\
118  -o create_board_charuco -lopencv_core -lopencv_imgcodecs -lopencv_highgui -lopencv_aruco -lopencv_imgproc -lopencv_videoio
119 
120  g++ -I/usr/share/jevois-opencv-3.2.0/include -L/usr/share/jevois-opencv-3.2.0/lib calibrate_camera_charuco.cpp \\
121  -o calibrate_camera_charuco -lopencv_core -lopencv_imgcodecs -lopencv_highgui -lopencv_aruco -lopencv_imgproc -lopencv_videoio
122  \endcode
123 
124  To create a ChArUco board that can be printed, using dictionary 0, and then derive the camera parameters from it
125  (set the `--ml` and `--sl` parameters to what you measure on your printed board; below are what we measured after
126  printing the 5x8 charuco board on US Letter paper with auto scaling/rotate to fit paper):
127 
128  \code
129  ./create_board_charuco -d=0 -h=5 -w=8 --ml=200 --sl=350 charuco.png
130  ./calibrate_camera_charuco -d=0 -h=5 -w=8 --ml=.0172 --sl=.0303 --rs --sc calibration.yaml
131  \endcode
132 
133  \ingroup components */
134 class ArUco : public jevois::Component,
135  public jevois::Parameter<aruco::camparams, aruco::detparams, aruco::dictionary,
136  aruco::dopose, aruco::markerlen, aruco::showcube>
137 {
138  public:
139  //! Constructor
141 
142  //! Destructor
143  virtual ~ArUco();
144 
145  //! Initialize, create the detector and read the config files
146  void postInit() override;
147 
148  //! Un-initialize, nuke allocated resources
149  void postUninit() override;
150 
151  //! Detect markers
152  void detectMarkers(cv::InputArray image, cv::OutputArray ids, cv::OutputArrayOfArrays corners);
153 
154  //! Estimate pose of individual markers
155  void estimatePoseSingleMarkers(cv::InputArrayOfArrays corners, cv::OutputArray rvecs, cv::OutputArray tvecs);
156 
157  //! Draw any markers previously detected by detectMarkers()
158  /*! If txtx,txty are positive, also print a text string there */
159  void drawDetections(jevois::RawImage & outimg, int txtx, int txty, std::vector<int> ids,
160  std::vector<std::vector<cv::Point2f> > corners, std::vector<cv::Vec3d> const & rvecs,
161  std::vector<cv::Vec3d> const & tvecs);
162 
163  //! Send serial messages about detections
164  /*! The module given should be the owner of this component, we will use it to actually send each serial message
165  using some variant of jevois::Module::sendSerial(). */
166  void sendSerial(jevois::StdModule * mod, std::vector<int> ids, std::vector<std::vector<cv::Point2f> > corners,
167  unsigned int w, unsigned int h, std::vector<cv::Vec3d> const & rvecs,
168  std::vector<cv::Vec3d> const & tvecs);
169 
170  //! Our current camera matrix
171  cv::Mat itsCamMatrix;
172 
173  //! Our current distortion coefficients
174  cv::Mat itsDistCoeffs;
175 
176  protected:
177  cv::Ptr<cv::aruco::DetectorParameters> itsDetectorParams;
178  cv::Ptr<cv::aruco::Dictionary> itsDictionary;
179 };
180 
JEVOIS_DECLARE_PARAMETER(thresh1, double, "First threshold for hysteresis", 50.0, ParamCateg)
Simple wrapper class over the opencv_contrib ArUco augmented reality markers.
Definition: ArUco.H:134
Definition: ArUco.H:25
cv::Mat itsDistCoeffs
Our current distortion coefficients.
Definition: ArUco.H:174
cv::Mat itsCamMatrix
Our current camera matrix.
Definition: ArUco.H:171
friend friend class Component
cv::Ptr< cv::aruco::Dictionary > itsDictionary
Definition: ArUco.H:178
JEVOIS_DEFINE_ENUM_CLASS(CameraSensor,(any)(ov9650)(ov2640)(ov7725)(ar0135))
cv::Ptr< cv::aruco::DetectorParameters > itsDetectorParams
Definition: ArUco.H:177