JeVoisBase
1.19
JeVois Smart Embedded Machine Vision Toolkit Base Modules
|
|
Simple example of object detection using ORB keypoints followed by 6D pose estimation in Python. More...
Public Member Functions | |
def | __init__ (self) |
Constructor. More... | |
def | loadCameraCalibration (self, w, h) |
Load camera calibration from JeVois share directory. More... | |
def | detect (self, imggray, outimg=None) |
Detect objects using keypoints. More... | |
def | estimatePose (self, hlist) |
Estimate 6D pose of each of the quadrilateral objects in hlist: More... | |
def | sendAllSerial (self, w, h, hlist, rvecs, tvecs) |
Send serial messages, one per object. More... | |
def | drawDetections (self, outimg, hlist, rvecs=None, tvecs=None) |
Draw all detected objects in 3D. More... | |
def | processNoUSB (self, inframe) |
Process function with no USB output. More... | |
def | process (self, inframe, outframe) |
Process function with USB output. More... | |
Public Attributes | |
fname | |
owm | |
ohm | |
wintop | |
winleft | |
winw | |
winh | |
distth | |
timer | |
camMatrix | |
distCoeffs | |
detector | |
refdes | |
refcorners | |
wincorners | |
matcher | |
Simple example of object detection using ORB keypoints followed by 6D pose estimation in Python.
This module implements an object detector using ORB keypoints using OpenCV in Python. Its main goal is to also demonstrate full 6D pose recovery of the detected object, in Python, as well as locating in 3D a sub-element of the detected object (here, a window within a larger textured wall). See ObjectDetect for more info about object detection using keypoints. This module is available with JeVois v1.6.3 and later.
The algorithm consists of 5 phases:
For more information about ORB keypoint detection and matching in OpenCV, see, e.g., https://docs.opencv.org/3.4.0/d1/d89/tutorial_py_orb.html
This module is provided for inspiration. It has no pretension of actually solving the FIRST Robotics Power Up (sm) vision problem in a complete and reliable way. It is released in the hope that FRC teams will try it out and get inspired to develop something much better for their own robot.
Note how, contrary to FirstVision, DemoArUco, etc, the green parallelepiped is drawn going into the object instead of sticking out of it, as it is depicting a tunnel at the window location.
This module is for now specific to the "exchange" of the FIRST Robotics 2018 Power Up (sm) challenge. See https://www.firstinspires.org/resource-library/frc/competition-manual-qa-system
The exchange is a large textured structure with a window at the bottom into which robots should deliver foam cubes.
A reference picture of the whole exchange (taken from the official rules) is in JEVOIS:/modules/JeVois/PythonObject6D/images/reference.png on your JeVois microSD card. It will be processed when the module starts. No additional training procedure is needed.
If you change the reference image, you should also edit:
self.owm
and self.ohm
to the width ahd height, in meters, of the actual physical object in your picture. Square pixels are assumed, so make sure the aspect ratio of your PNG image matches the aspect ratio in meters given by variables self.owm
and self.ohm
in the code.self.wintop
, self.winleft
, self.winw
, self.winh
to the location of the top-left corner, in meters and relative to the top-left corner of the whole reference object, of a window of interest (the tunnel into which the cubes should be delivered), and width and height, in meters, of the window.TODO: Add support for multiple images and online training as in ObjectDetect
There are a number of limitations and caveats to this module:
Definition at line 110 of file PythonObject6D.py.
def PythonObject6D.PythonObject6D.__init__ | ( | self | ) |
Constructor.
Definition at line 113 of file PythonObject6D.py.
def PythonObject6D.PythonObject6D.detect | ( | self, | |
imggray, | |||
outimg = None |
|||
) |
Detect objects using keypoints.
Definition at line 150 of file PythonObject6D.py.
References hasattr().
Referenced by PythonObject6D.PythonObject6D.process(), and PythonObject6D.PythonObject6D.processNoUSB().
def PythonObject6D.PythonObject6D.drawDetections | ( | self, | |
outimg, | |||
hlist, | |||
rvecs = None , |
|||
tvecs = None |
|||
) |
Draw all detected objects in 3D.
Definition at line 293 of file PythonObject6D.py.
References FirstPython.FirstPython.camMatrix, PythonObject6D.PythonObject6D.camMatrix, FirstPython.FirstPython.distCoeffs, PythonObject6D.PythonObject6D.distCoeffs, FirstVision::SinglePoseEstimationParallel.distCoeffs, demo.int, PythonObject6D.PythonObject6D.winh, and PythonObject6D.PythonObject6D.winw.
Referenced by PythonObject6D.PythonObject6D.process().
def PythonObject6D.PythonObject6D.estimatePose | ( | self, | |
hlist | |||
) |
Estimate 6D pose of each of the quadrilateral objects in hlist:
Definition at line 245 of file PythonObject6D.py.
References FirstPython.FirstPython.camMatrix, PythonObject6D.PythonObject6D.camMatrix, FirstPython.FirstPython.distCoeffs, PythonObject6D.PythonObject6D.distCoeffs, FirstVision::SinglePoseEstimationParallel.distCoeffs, PythonObject6D.PythonObject6D.winh, and PythonObject6D.PythonObject6D.winw.
Referenced by PythonObject6D.PythonObject6D.process(), and PythonObject6D.PythonObject6D.processNoUSB().
def PythonObject6D.PythonObject6D.loadCameraCalibration | ( | self, | |
w, | |||
h | |||
) |
Load camera calibration from JeVois share directory.
Definition at line 136 of file PythonObject6D.py.
Referenced by PythonObject6D.PythonObject6D.process(), and PythonObject6D.PythonObject6D.processNoUSB().
def PythonObject6D.PythonObject6D.process | ( | self, | |
inframe, | |||
outframe | |||
) |
Process function with USB output.
Definition at line 392 of file PythonObject6D.py.
References ObjectMatcher.detect(), BlobDetector.detect(), FirstPython.FirstPython.detect(), PythonObject6D.PythonObject6D.detect(), FirstVision.detect(), QRcode.drawDetections(), ARtoolkit.drawDetections(), Yolo.drawDetections(), ArUco.drawDetections(), PythonObject6D.PythonObject6D.drawDetections(), FirstPython.FirstPython.drawDetections(), FirstVision.drawDetections(), PythonObject6D.PythonObject6D.estimatePose(), FirstPython.FirstPython.estimatePose(), FirstVision.estimatePose(), hasattr(), FirstPython.FirstPython.loadCameraCalibration(), PythonObject6D.PythonObject6D.loadCameraCalibration(), FirstVision.loadCameraCalibration(), PythonObject6D.PythonObject6D.sendAllSerial(), FirstPython.FirstPython.sendAllSerial(), FirstVision.sendAllSerial(), PythonTutorial2::PythonTutorial2.timer, demangling.timer, PyFaceMesh.PyFaceMesh.timer, PyHandDetector.PyHandDetector.timer, PyPoseDetector.PyPoseDetector.timer, PyDMTX.PyDMTX.timer, PySelfie.PySelfie.timer, PyObjectron.PyObjectron.timer, PyEmotion.PyEmotion.timer, PyLicensePlate.PyLicensePlate.timer, PyCoralDetect.PyCoralDetect.timer, PyCoralSegment.PyCoralSegment.timer, PySceneText.PySceneText.timer, PyCoralClassify.PyCoralClassify.timer, PyClassificationDNN.PyClassificationDNN.timer, FirstPython.FirstPython.timer, PyDetectionDNN.PyDetectionDNN.timer, and PythonObject6D.PythonObject6D.timer.
def PythonObject6D.PythonObject6D.processNoUSB | ( | self, | |
inframe | |||
) |
Process function with no USB output.
Definition at line 367 of file PythonObject6D.py.
References ObjectMatcher.detect(), BlobDetector.detect(), FirstPython.FirstPython.detect(), PythonObject6D.PythonObject6D.detect(), FirstVision.detect(), PythonObject6D.PythonObject6D.estimatePose(), FirstPython.FirstPython.estimatePose(), FirstVision.estimatePose(), hasattr(), FirstPython.FirstPython.loadCameraCalibration(), PythonObject6D.PythonObject6D.loadCameraCalibration(), FirstVision.loadCameraCalibration(), PythonObject6D.PythonObject6D.sendAllSerial(), FirstPython.FirstPython.sendAllSerial(), FirstVision.sendAllSerial(), PythonTutorial2::PythonTutorial2.timer, demangling.timer, PyFaceMesh.PyFaceMesh.timer, PyHandDetector.PyHandDetector.timer, PyPoseDetector.PyPoseDetector.timer, PyDMTX.PyDMTX.timer, PySelfie.PySelfie.timer, PyObjectron.PyObjectron.timer, PyEmotion.PyEmotion.timer, PyLicensePlate.PyLicensePlate.timer, PyCoralDetect.PyCoralDetect.timer, PyCoralSegment.PyCoralSegment.timer, PySceneText.PySceneText.timer, PyCoralClassify.PyCoralClassify.timer, PyClassificationDNN.PyClassificationDNN.timer, FirstPython.FirstPython.timer, PyDetectionDNN.PyDetectionDNN.timer, and PythonObject6D.PythonObject6D.timer.
def PythonObject6D.PythonObject6D.sendAllSerial | ( | self, | |
w, | |||
h, | |||
hlist, | |||
rvecs, | |||
tvecs | |||
) |
Send serial messages, one per object.
Definition at line 269 of file PythonObject6D.py.
References FirstPython.FirstPython.ohm, PythonObject6D.PythonObject6D.ohm, FirstPython.FirstPython.owm, and PythonObject6D.PythonObject6D.owm.
Referenced by PythonObject6D.PythonObject6D.process(), and PythonObject6D.PythonObject6D.processNoUSB().
PythonObject6D.PythonObject6D.camMatrix |
Definition at line 140 of file PythonObject6D.py.
Referenced by PythonObject6D.PythonObject6D.drawDetections(), and PythonObject6D.PythonObject6D.estimatePose().
PythonObject6D.PythonObject6D.detector |
Definition at line 156 of file PythonObject6D.py.
PythonObject6D.PythonObject6D.distCoeffs |
Definition at line 141 of file PythonObject6D.py.
Referenced by PythonObject6D.PythonObject6D.drawDetections(), and PythonObject6D.PythonObject6D.estimatePose().
PythonObject6D.PythonObject6D.distth |
Definition at line 129 of file PythonObject6D.py.
PythonObject6D.PythonObject6D.fname |
Definition at line 115 of file PythonObject6D.py.
PythonObject6D.PythonObject6D.matcher |
Definition at line 179 of file PythonObject6D.py.
PythonObject6D.PythonObject6D.ohm |
Definition at line 119 of file PythonObject6D.py.
Referenced by PythonObject6D.PythonObject6D.sendAllSerial().
PythonObject6D.PythonObject6D.owm |
Definition at line 118 of file PythonObject6D.py.
Referenced by PythonObject6D.PythonObject6D.sendAllSerial().
PythonObject6D.PythonObject6D.refcorners |
Definition at line 165 of file PythonObject6D.py.
PythonObject6D.PythonObject6D.refdes |
Definition at line 161 of file PythonObject6D.py.
PythonObject6D.PythonObject6D.timer |
Definition at line 132 of file PythonObject6D.py.
Referenced by PythonOpenCV.PythonOpenCV.process(), PythonSandbox.PythonSandbox.process(), PythonParallel.PythonParallel.process(), PythonObject6D.PythonObject6D.process(), PythonTest.PythonTest.processGUI(), PythonSandbox.PythonSandbox.processGUI(), PythonSandbox.PythonSandbox.processNoUSB(), and PythonObject6D.PythonObject6D.processNoUSB().
PythonObject6D.PythonObject6D.wincorners |
Definition at line 166 of file PythonObject6D.py.
PythonObject6D.PythonObject6D.winh |
Definition at line 126 of file PythonObject6D.py.
Referenced by PythonObject6D.PythonObject6D.drawDetections(), and PythonObject6D.PythonObject6D.estimatePose().
PythonObject6D.PythonObject6D.winleft |
Definition at line 124 of file PythonObject6D.py.
PythonObject6D.PythonObject6D.wintop |
Definition at line 123 of file PythonObject6D.py.
PythonObject6D.PythonObject6D.winw |
Definition at line 125 of file PythonObject6D.py.
Referenced by PythonObject6D.PythonObject6D.drawDetections(), and PythonObject6D.PythonObject6D.estimatePose().