ArUco detection and orientation module

Question

Hi, 

First, congrats for developing this amazing camera, it's gonna be very useful for small robotics projects.

I try to use the object detection module to control the orientation of the camera and follow the tracked object. First results are quite impressive: https://www.youtube.com/playlist?list=PL5oWZ1hXvfPUtm_PJMP9iwe9fPo74-KJM .

Ultimately, i would need the sets of orientations (and translations) of the camera in respect with an object. The OpenCV ArUco module would be an excellent option for this (http://docs.opencv.org/3.1.0/d5/dae/tutorial_aruco_detection.html ; https://www.youtube.com/watch?v=IsXWrcB_Hvs)

Since i have only few experience in C++, i wondered in anyone tried (or would like to try) to install this module on the JeVois camera in order to detect the ArUcos and send in serial their respective rotations and translations.

Any tips are welcome, 
Thanks!

Answer 1

Great work with that pan/tilt head!

Yes, ArUco is great and is already supported by JeVois: http://jevois.org/moddoc/DemoArUco/modinfo.html

Two caveats that should be relatively easy to fix:

- the serial strings are not of the form T2D X Y like those of the ObjectTracker, this is because one may often want to detect several ArUco markers and get their ID in addition to location. You can either modify the Arduino code to parse the serial strings of the ArUco module, of change the ArUco module to output T2D strings.

- in principle, once you calibrate your camera, you can get the full 6D pose of the marker (3D position and 3D orientation). That code used to work in OpenCV 3.1 but something changed in OpenCV when upgrading to OpenCV 3.2 and we have not had time to check it out yet. So when you turn the showpose parameter in our JeVois ArUco module to true, you will get some errors about OpenCV not liking the objects we pass to the pose estimation algorithm, we need to better understand what the pose estimation now wants in OpenCV 3.2 so we can send it the correct data. Since it just uses an InputArraysOfArrays it is not clear to me what exactly should be inside that, and apparently it is not what it used to be with OpenC 3.1. Also, I am unclear about how exactly the calibration is supposed to work, it requires some measurements like marker size and spacing between markers but I am not sure exactly what those correspond to. See the section on recovering 3D pose at http://jevois.org/basedoc/classArUco.html#details

Comments

Doing camera calibration using ArUco isn't much different than doing it the standard OpenCV way - you print a calibration pattern like a checkerboard and take lots of photos of the pattern at different positions in the camera view. But in this case you use a chArUcoboard pattern, which is a checkerboard with ArUco markers in each of the white squares (and the size/spacing of those is what you need to provide.) That gives the calibration routine more information about exactly how the checkerboard is oriented, and means you can use pictures where the pattern is partially obscured or out of frame, which isn't possible with the normal method.

It's described in the tutorials that come with OpenCV: http://docs.opencv.org/master/da/d13/tutorial_aruco_calibration.html

---

guys I finally found some time to fix this, latest JeVois ArUco code in GitHub supports 3D pose estimation and display, as well as sending 3D location and pose over serial. The location component seems ok but I need to check the pose quaternion. I will build a robot arm rig to try this out.

---

Would you mind adding a parameter in the string output for the marker orientation (even in 2D) ? I guess we can already compute it from quaternions but it would be quite good to have it ! It shouldn't need dopose to be set to true to get only the orientation around the Z axis.