Simple example of FIRST Robotics image processing pipeline using OpenCV in Python on JeVois. More...

Public Member Functions
	__init__ (self)
	Constructor.

	loadCameraCalibration (self, w, h)
	Load camera calibration from JeVois share directory.

	detect (self, imgbgr, outimg=None)
	Detect objects within our HSV range.

	estimatePose (self, hlist)
	Estimate 6D pose of each of the quadrilateral objects in hlist:

	sendAllSerial (self, w, h, hlist, rvecs, tvecs)
	Send serial messages, one per object.

	drawDetections (self, outimg, hlist, rvecs=None, tvecs=None)
	Draw all detected objects in 3D.

	processNoUSB (self, inframe)
	Process function with no USB output.

	process (self, inframe, outframe)
	Process function with USB output.

Public Attributes
	HSVmin

	HSVmax

	owm

	ohm

	epsilon

	hullarea

	hullfill

	ethresh

	margin

	timer

	camMatrix

	distCoeffs

	erodeElement

	dilateElement

Detailed Description

Simple example of FIRST Robotics image processing pipeline using OpenCV in Python on JeVois.

This module is a simplified version of the C++ module FirstVision. It is available with JeVois v1.6.2 or later.

This module implements a simple color-based object detector using OpenCV in Python. Its main goal is to also demonstrate full 6D pose recovery of the detected object, in Python.

This module isolates pixels within a given HSV range (hue, saturation, and value of color pixels), does some cleanups, and extracts object contours. It is looking for a rectangular U shape of a specific size (set by parameters owm and ohm for object width and height in meters). See screenshots for an example of shape. It sends information about detected objects over serial.

This module usually works best with the camera sensor set to manual exposure, manual gain, manual color balance, etc so that HSV color values are reliable. See the file script.cfg file in this module's directory for an example of how to set the camera settings each time this module is loaded.

This module is provided for inspiration. It has no pretension of actually solving the FIRST Robotics vision problem in a complete and reliable way. It is released in the hope that FRC teams will try it out and get inspired to develop something much better for their own robot.

Using this module

Check out this tutorial first, for the FirstVision module written in C++ and also check out the doc for FirstVision. Then you can just dive in and start editing the python code of FirstPython.

See http://jevois.org/tutorials for tutorials on getting started with programming JeVois in Python without having to install any development software on your host computer.

Trying it out

Edit the module's file at JEVOIS:/modules/JeVois/FirstPython/FirstPython.py and set the parameters self.owm and self.ohm to the physical width and height of your U-shaped object in meters. You should also review and edit the other parameters in the module's constructor, such as the range of HSV colors.

Author: Laurent Itti

Display Name:: FIRST Python

Videomapping:: YUYV 640 252 60.0 YUYV 320 240 60.0 JeVois FirstPython

Videomapping:: YUYV 320 252 60.0 YUYV 320 240 60.0 JeVois FirstPython

Email:: itti@usc.edu

Address:: University of Southern California, HNB-07A, 3641 Watt Way, Los Angeles, CA 90089-2520, USA

Copyright: Copyright (C) 2018 by Laurent Itti, iLab and the University of Southern California

Main URL:: http://jevois.org

Support URL:: http://jevois.org/doc

Other URL:: http://iLab.usc.edu

License:: GPL v3

Distribution:: Unrestricted

Restrictions:: None

Definition at line 77 of file FirstPython.py.

Constructor & Destructor Documentation

◆ init()

FirstPython.FirstPython.__init__ ( self )

Constructor.

Definition at line 80 of file FirstPython.py.

Member Function Documentation

◆ detect()

FirstPython.FirstPython.detect	(	self,
		imgbgr,
		outimg = `None`
	)

Detect objects within our HSV range.

Definition at line 121 of file FirstPython.py.

References FirstPython.FirstPython.HSVmax, and FirstPython.FirstPython.HSVmin.

Referenced by FirstPython.FirstPython.process(), PythonObject6D.PythonObject6D.process(), FirstPython.FirstPython.processNoUSB(), and PythonObject6D.PythonObject6D.processNoUSB().

◆ drawDetections()

FirstPython.FirstPython.drawDetections	(	self,
		outimg,
		hlist,
		rvecs = `None`,
		tvecs = `None`
	)

Draw all detected objects in 3D.

Definition at line 300 of file FirstPython.py.

References jevois::CameraCalibration.camMatrix, FirstPython.FirstPython.camMatrix, PythonObject6D.PythonObject6D.camMatrix, jevois::CameraCalibration.distCoeffs, FirstPython.FirstPython.distCoeffs, FirstVision::SinglePoseEstimationParallel.distCoeffs, PythonObject6D.PythonObject6D.distCoeffs, FirstPython.FirstPython.ohm, PythonObject6D.PythonObject6D.ohm, FirstPython.FirstPython.owm, and PythonObject6D.PythonObject6D.owm.

Referenced by FirstPython.FirstPython.process(), and PythonObject6D.PythonObject6D.process().

◆ estimatePose()

FirstPython.FirstPython.estimatePose	(	self,
		hlist
	)

Estimate 6D pose of each of the quadrilateral objects in hlist:

Definition at line 252 of file FirstPython.py.

References jevois::CameraCalibration.camMatrix, FirstPython.FirstPython.camMatrix, PythonObject6D.PythonObject6D.camMatrix, jevois::CameraCalibration.distCoeffs, FirstPython.FirstPython.distCoeffs, FirstVision::SinglePoseEstimationParallel.distCoeffs, PythonObject6D.PythonObject6D.distCoeffs, FirstPython.FirstPython.ohm, PythonObject6D.PythonObject6D.ohm, FirstPython.FirstPython.owm, and PythonObject6D.PythonObject6D.owm.

Referenced by FirstPython.FirstPython.process(), PythonObject6D.PythonObject6D.process(), FirstPython.FirstPython.processNoUSB(), and PythonObject6D.PythonObject6D.processNoUSB().

◆ loadCameraCalibration()

FirstPython.FirstPython.loadCameraCalibration	(	self,
		w,
		h
	)

Load camera calibration from JeVois share directory.

Definition at line 110 of file FirstPython.py.

Referenced by FirstPython.FirstPython.process(), PythonObject6D.PythonObject6D.process(), FirstPython.FirstPython.processNoUSB(), and PythonObject6D.PythonObject6D.processNoUSB().

◆ process()

FirstPython.FirstPython.process	(	self,
		inframe,
		outframe
	)

Process function with USB output.

Definition at line 392 of file FirstPython.py.

References ObjectMatcher.detect(), BlobDetector.detect(), FirstVision.detect(), FirstPython.FirstPython.detect(), PythonObject6D.PythonObject6D.detect(), ARtoolkit.drawDetections(), ArUco.drawDetections(), QRcode.drawDetections(), Yolo.drawDetections(), ArUco.drawDetections(), QRcode.drawDetections(), ARtoolkit.drawDetections(), FirstVision.drawDetections(), FirstPython.FirstPython.drawDetections(), PythonObject6D.PythonObject6D.drawDetections(), FirstPython.FirstPython.estimatePose(), PythonObject6D.PythonObject6D.estimatePose(), FirstVision.estimatePose(), FirstPython.FirstPython.loadCameraCalibration(), PythonObject6D.PythonObject6D.loadCameraCalibration(), jevois::Engine.loadCameraCalibration(), FirstVision.sendAllSerial(), FirstPython.FirstPython.sendAllSerial(), PythonObject6D.PythonObject6D.sendAllSerial(), AprilTag.AprilTag.timer, FirstPython.FirstPython.timer, PyClassificationDNN.PyClassificationDNN.timer, PyCoralClassify.PyCoralClassify.timer, PyCoralDetect.PyCoralDetect.timer, PyCoralSegment.PyCoralSegment.timer, PyDetectionDNN.PyDetectionDNN.timer, PyDMTX.PyDMTX.timer, PyEmotion.PyEmotion.timer, PyFaceMesh.PyFaceMesh.timer, PyHandDetector.PyHandDetector.timer, PyLicensePlate.PyLicensePlate.timer, PyObjectron.PyObjectron.timer, PyPoseDetector.PyPoseDetector.timer, PySceneText.PySceneText.timer, PySelfie.PySelfie.timer, PythonObject6D.PythonObject6D.timer, PythonOpenCV.PythonOpenCV.timer, PythonParallel.PythonParallel.timer, PythonSandbox.PythonSandbox.timer, and PythonTest.PythonTest.timer.

◆ processNoUSB()

FirstPython.FirstPython.processNoUSB	(	self,
		inframe
	)

Process function with no USB output.

Definition at line 367 of file FirstPython.py.

References ObjectMatcher.detect(), BlobDetector.detect(), FirstVision.detect(), FirstPython.FirstPython.detect(), PythonObject6D.PythonObject6D.detect(), FirstPython.FirstPython.estimatePose(), PythonObject6D.PythonObject6D.estimatePose(), FirstVision.estimatePose(), FirstPython.FirstPython.loadCameraCalibration(), PythonObject6D.PythonObject6D.loadCameraCalibration(), jevois::Engine.loadCameraCalibration(), FirstVision.sendAllSerial(), FirstPython.FirstPython.sendAllSerial(), PythonObject6D.PythonObject6D.sendAllSerial(), AprilTag.AprilTag.timer, FirstPython.FirstPython.timer, PyClassificationDNN.PyClassificationDNN.timer, PyCoralClassify.PyCoralClassify.timer, PyCoralDetect.PyCoralDetect.timer, PyCoralSegment.PyCoralSegment.timer, PyDetectionDNN.PyDetectionDNN.timer, PyDMTX.PyDMTX.timer, PyEmotion.PyEmotion.timer, PyFaceMesh.PyFaceMesh.timer, PyHandDetector.PyHandDetector.timer, PyLicensePlate.PyLicensePlate.timer, PyObjectron.PyObjectron.timer, PyPoseDetector.PyPoseDetector.timer, PySceneText.PySceneText.timer, PySelfie.PySelfie.timer, PythonObject6D.PythonObject6D.timer, PythonOpenCV.PythonOpenCV.timer, PythonParallel.PythonParallel.timer, PythonSandbox.PythonSandbox.timer, and PythonTest.PythonTest.timer.

◆ sendAllSerial()

FirstPython.FirstPython.sendAllSerial	(	self,
		w,
		h,
		hlist,
		rvecs,
		tvecs
	)

Send serial messages, one per object.

Definition at line 276 of file FirstPython.py.

References FirstPython.FirstPython.ohm, PythonObject6D.PythonObject6D.ohm, FirstPython.FirstPython.owm, and PythonObject6D.PythonObject6D.owm.

Referenced by FirstPython.FirstPython.process(), PythonObject6D.PythonObject6D.process(), FirstPython.FirstPython.processNoUSB(), and PythonObject6D.PythonObject6D.processNoUSB().

Member Data Documentation

◆ camMatrix

FirstPython.FirstPython.camMatrix

Definition at line 112 of file FirstPython.py.

Referenced by FirstPython.FirstPython.drawDetections(), PythonObject6D.PythonObject6D.drawDetections(), FirstPython.FirstPython.estimatePose(), and PythonObject6D.PythonObject6D.estimatePose().

◆ dilateElement

FirstPython.FirstPython.dilateElement

Definition at line 136 of file FirstPython.py.

◆ distCoeffs

FirstPython.FirstPython.distCoeffs

Definition at line 112 of file FirstPython.py.

Referenced by FirstPython.FirstPython.drawDetections(), PythonObject6D.PythonObject6D.drawDetections(), FirstPython.FirstPython.estimatePose(), and PythonObject6D.PythonObject6D.estimatePose().

◆ epsilon

FirstPython.FirstPython.epsilon

Definition at line 95 of file FirstPython.py.

◆ erodeElement

FirstPython.FirstPython.erodeElement

Definition at line 135 of file FirstPython.py.

◆ ethresh

FirstPython.FirstPython.ethresh

Definition at line 98 of file FirstPython.py.

◆ HSVmax

FirstPython.FirstPython.HSVmax

Definition at line 88 of file FirstPython.py.

Referenced by FirstPython.FirstPython.detect().

◆ HSVmin

FirstPython.FirstPython.HSVmin

Definition at line 87 of file FirstPython.py.

Referenced by FirstPython.FirstPython.detect().

◆ hullarea

FirstPython.FirstPython.hullarea

Definition at line 96 of file FirstPython.py.

◆ hullfill

FirstPython.FirstPython.hullfill

Definition at line 97 of file FirstPython.py.

◆ margin

FirstPython.FirstPython.margin

Definition at line 99 of file FirstPython.py.

◆ ohm

FirstPython.FirstPython.ohm

Definition at line 92 of file FirstPython.py.

Referenced by FirstPython.FirstPython.drawDetections(), FirstPython.FirstPython.estimatePose(), FirstPython.FirstPython.sendAllSerial(), and PythonObject6D.PythonObject6D.sendAllSerial().

◆ owm

FirstPython.FirstPython.owm

Definition at line 91 of file FirstPython.py.

Referenced by FirstPython.FirstPython.drawDetections(), FirstPython.FirstPython.estimatePose(), FirstPython.FirstPython.sendAllSerial(), and PythonObject6D.PythonObject6D.sendAllSerial().

◆ timer

FirstPython.FirstPython.timer

Definition at line 102 of file FirstPython.py.

Referenced by FirstPython.FirstPython.process(), PyClassificationDNN.PyClassificationDNN.process(), PyCoralClassify.PyCoralClassify.process(), PyCoralDetect.PyCoralDetect.process(), PyCoralSegment.PyCoralSegment.process(), PyDetectionDNN.PyDetectionDNN.process(), PyDMTX.PyDMTX.process(), PythonObject6D.PythonObject6D.process(), PythonOpenCV.PythonOpenCV.process(), PythonParallel.PythonParallel.process(), PythonSandbox.PythonSandbox.process(), AprilTag.AprilTag.processGUI(), PyCoralClassify.PyCoralClassify.processGUI(), PyCoralDetect.PyCoralDetect.processGUI(), PyCoralSegment.PyCoralSegment.processGUI(), PyFaceMesh.PyFaceMesh.processGUI(), PyHandDetector.PyHandDetector.processGUI(), PyLicensePlate.PyLicensePlate.processGUI(), PyObjectron.PyObjectron.processGUI(), PyPoseDetector.PyPoseDetector.processGUI(), PySceneText.PySceneText.processGUI(), PySelfie.PySelfie.processGUI(), PythonOpenCV.PythonOpenCV.processGUI(), PythonSandbox.PythonSandbox.processGUI(), PythonTest.PythonTest.processGUI(), FirstPython.FirstPython.processNoUSB(), PythonObject6D.PythonObject6D.processNoUSB(), and PythonSandbox.PythonSandbox.processNoUSB().

The documentation for this class was generated from the following file:

src/Modules/FirstPython/FirstPython.py

Public Member Functions

Public Attributes

Detailed Description

Using this module

Trying it out

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ detect()

◆ drawDetections()

◆ estimatePose()

◆ loadCameraCalibration()

◆ process()

◆ processNoUSB()

◆ sendAllSerial()

Member Data Documentation

◆ camMatrix

◆ dilateElement

◆ distCoeffs

◆ epsilon

◆ erodeElement

◆ ethresh

◆ HSVmax

◆ HSVmin

◆ hullarea

◆ hullfill

◆ margin

◆ ohm

◆ owm

◆ timer

◆ init()