JeVois Tutorials
1.21
JeVois Smart Embedded Machine Vision Tutorials
|
|
Several JeVois machine vision modules detect and locate target objects in video, either based on their visual salience, on a specific color, on a specific type of shape (e.g., ArUco marker), or on a generic learned appearance. Some of these modules issue messages to the JeVois serial ports that indicate the coordinates of the objects that were found (and adding such messages to those which do not yet send them is a simple modification).
In this tutorial, we use a motorized pan-tilt head and a small Arduino-compatible board to control the pan-tilt head, so that it will always try to center the detected object in the camera's field of view. That is, we will use the coordinates issued by JeVois, to counter-rotate the pan-tilt head (onto which JeVois should be mounted) so as to center the objects of interest. The net effect will be that JeVois mounted on the pan-tilt head will track the objects of interest.
This tutorial is a direct extension, with more details, step-by-step instructions, and with pictures, of the Tutorial on how to write Arduino code that interacts with JeVois in the main JeVois documentation.
For this tutorial, we will use:
serout
messages to us and serlog
messages to nobody.Depending on the construction of your pan-tilt head, increasing the pan angle may turn the head rightwards or leftwards, and, likewise, increasing the tilt angle could turn the head up or down. To figure this out, we will first connect to the pan-tilt head manually. We will send hand-written commands to the head.
Before you start this, make sure you understand the following:
serout
and serlog
parametersserout
and serlog
commandsNow let's try the following:
setpar serout Hard
Now let's move the pan-tilt head by hand. By default, it homes to standardized coordinates (0, 0). If we ask it to move to, say, (1000, 0), it should turn rightward. We will do that by using the serout
command, which just forwards everything that is passed to it directly to the serout port. Hence, everything we type in a serout
command will be sent to our Arduino.
Note that the Arduino pan-tilt code is written to apply a small relative offset to a memorized pan-tilt location each time a serial command is received. Indeed, because the Arduino code will receive target coordinates up to 60 times per second, it would be too jerky to try each time to fully home onto the target. So, instead, the pan-tilt head will take a small step towards the target. The size of these steps is regulated by two control parameters in the Arduino code, PANGAIN
and TILTGAIN
, with default values of 100. Higher gain values will move the head more rapidly towards the target on each video frame.
serout T2 10000 0
As the command is received, the pan-tilt head should turn right by some amount. If it turns left, your pan gain should be negated. The Arduino code supports two commands, in addition to T2: PANGAIN
and TILTGAIN
. You can negate PANGAIN
as follows if your pan-tilt head just turned left:
serout PANGAIN -100
Then try again
serout T2 10000 0
and the camera should now pan in the other direction than previously.
serout T2 0 10000
to tilt the camera up, and
serout TILTGAIN -100
Once you are done playing, make a note of what you did and you can later put these commands in a script.cfg file in the directory of the machine vision module you will be using (see how we did that in Tutorial on how to write Arduino code that interacts with JeVois).
We are ready to get that pan-tilt head controlled by JeVois.
You may want to increase the pan and tilt gains, which are very low by default. Higher values will move the pan-tilt head faster, so that it can better track fast-moving objects. The downside is that noisy inputs will also give rise to high jerky motions of the pan-tilt head.
Try the following:
serout PANGAIN 500
(or use -500 if you need a negative gain for your particular pan-tilt head). See how much faster the head can track now.
Once you are comfortable with the operation of the algorithm, you can also try it as a true standalone machine vision system with no USB output. The steps are as follows:
Supports mappings with NO USB output: YesAn example video mapping with no USB output (with NONE as USB pixel format) is also suggested:
Video Mapping: NONE 0 0 0.0 YUYV 320 240 60.0 JeVois ObjectTracker
setmapping2 YUYV 320 240 30.0 JeVois ObjectTracker setpar serout Hard streamonwhich will load up the object tracker module and run it in the mode with no USB output. The
setmapping2
command instructs JeVois to configure a given camera pixel format, resolution, and framerate, and to load the module ObjectTracker from Vendor JeVois. Because setmapping2
does not specify a USB output video format, the module will run with no USB video output. We then manually initiate video streaming using the streamon
command when running with no USB output, because there is no video capture software that is going to initiate it. See Command-line interface user guide for details on setmapping2
and streamon
.usbsd
in the command line interface, and a virtual USB flash drive named JEVOIS should appear on your host computer. See MicroSD card organization and files for details, towards bottom of the page.setmapping2 YUYV 320 240 30.0 JeVois ObjectTracker setpar serout Hard streamoninitscript.cfg is executed by JeVois when it boots up. You can put in there any command that is valid for the command-line interface of JeVois.
setmapping2
command, to match what we had before. You can then edit JEVOIS:/config/videomapping.cfg to change the framerate of the mode with USB output: Search for ObjectTracker and change this entry: YUYV 320 254 60.0 YUYV 320 240 60.0 JeVois ObjectTrackerto
YUYV 320 254 30.0 YUYV 320 240 30.0 JeVois ObjectTrackerand see User guide to video modes and mappings for details about the format of video mappings. Next time you start your video capture software, be sure to select YUYV 320x254 at 30 fps. Then have a look at Tuning the color-based object tracker using a python graphical interface for tuning. You can save your tuned parameters to the file JEVOIS:/modules/JeVois/ObjectTracker/script.cfg as we did in this tutorial.
PANGAIN
and TITLGAIN
values. Notice how values that are too high will yield crazy oscillations of the pan-tilt head. Indeed, we only use a proportional control of the head. As an exercise, think about how you would modify the Arduino code to include a PID control (see for example https://en.wikipedia.org/wiki/PID_controller).