Tuning the color-based object tracker using a python graphical interface

The color-based object tracker of JeVois is a simple but fast and effective way of tracking objects defined by a specific color. The behavior of the module can be fine-tuned by setting ranges for the hue (color tint), saturation (color fade), and value (color brightness) of the objects to track. While this can be achieved by hand, by using the approach studied in Changing machine vision parameters, quickly one becomes tired of typing many setpar commands while searching for appropriate settings of the 6 color-related parameters (hue min and max, saturation min and max, and value min and max).

Note

Note that since June 2018 it may be easier to use JeVois-A33: JeVois Inventor graphical user interface. The instructions below do not require JeVois Inventor are are provided in case you do not want to use the Inventor.

Here we write a simple graphical user interface in python to tune the parameters.

Hence, in this tutorial you will learn:

• How to tune the ObjectTracker module to track any color you like.
• How to communicate with JeVois from a python script.
• How to develop a simple python GUI with sliders that update JeVois parameters when they are modified.

Getting started

• This tutorial assumes a Linux host computer.
• Connect JeVois to a host computer.
• Start video capture software and select YUYV 320x254 @ 60.0 fps which should launch the ObjectTracker
• Start a serial terminal and connect to the serial-over-USB port of JeVois.

Available parameters

Typing help in your serial terminal or inspecting the documentation of Object Tracker reveals several parameters. Here we will focus on the hrange, srange, and vrange parameters, which define the range of hue, saturation, and value that we want to track.

If you are not familiar with the HSV color space, check out this Wikipedia page.

Each of these three parameters is a range, that is, we can change them by specifying both the minimum and maximum values, separated by ..., for example, try:

setpar hrange 10...200

A more user-friendly way

We write a simple python script that creates some sliders that one can drag with the mouse. When the sliders are updated, we send the appropriate setpar command to JeVois. Consider this starter python script, which is located in ~/jevois-tutorials/doc/snip/objectrackertuning.py:

#!/usr/bin/python
 
# Needed packages: sudo apt-get install python python-tk python-serial
 
# This tutorial is a simple program that allows one to adjust the hue, saturation, and value ranges of the ObjectTracker
# module using sliders
 
serdev = '/dev/ttyACM0' # serial device of JeVois
 
from Tkinter import *
import serial
import time
 
 
# default values for Hue, Saturation, and Value ranges:
hmin = 95
hmax = 110
smin = 100
smax = 255
vmin = 60
vmax = 253
 
####################################################################################################
# Send a command to JeVois and show response
def send_command(cmd):
    print "HOST>> " + cmd
    ser.write(cmd + '\n')
    out = ''
    time.sleep(0.1)
    while ser.inWaiting() > 0:
        out += ser.read(1)
    if out != '':
        print "JEVOIS>> " + out, # the final comma suppresses extra newline, since JeVois already sends one
        
####################################################################################################
def update_hmin(val):
    global hmin
    global hmax
    hmin = val
    send_command('setpar hrange {0}...{1}'.format(hmin, hmax))
    
####################################################################################################
def update_hmax(val):
    global hmin
    global hmax
    hmax = val
    send_command('setpar hrange {0}...{1}'.format(hmin, hmax))
    
####################################################################################################
def update_smin(val):
    global smin
    global smax
    smin = val
    send_command('setpar srange {0}...{1}'.format(smin, smax))
    
####################################################################################################
def update_smax(val):
    global smin
    global smax
    smax = val
    send_command('setpar srange {0}...{1}'.format(smin, smax))
 
####################################################################################################
def update_vmin(val):
    global vmin
    global vmax
    vmin = val
    send_command('setpar vrange {0}...{1}'.format(vmin, vmax))
    
####################################################################################################
def update_vmax(val):
    global vmin
    global vmax
    vmax = val
    send_command('setpar vrange {0}...{1}'.format(vmin, vmax))
    
####################################################################################################
# Main code
ser = serial.Serial(serdev, 115200, timeout=1)
send_command('ping')                   # should return ALIVE
 
master = Tk()
 
w1 = Label(master, text = "Hue min")
w1.pack()
w2 = Scale(master, from_=0, to=255, tickinterval=32, length=600, orient=HORIZONTAL, command=update_hmin)
w2.set(hmin)
w2.pack()
 
w3 = Label(master, text = "Hue max")
w3.pack()
w4 = Scale(master, from_=0, to=255, tickinterval=32, length=600, orient=HORIZONTAL, command=update_hmax)
w4.set(hmax)
w4.pack()
 
w5 = Label(master, text = "Saturation min")
w5.pack()
w6 = Scale(master, from_=0, to=255, tickinterval=32, length=600, orient=HORIZONTAL, command=update_smin)
w6.set(smin)
w6.pack()
 
w7 = Label(master, text = "Saturation max")
w7.pack()
w8 = Scale(master, from_=0, to=255, tickinterval=32, length=600, orient=HORIZONTAL, command=update_smax)
w8.set(smax)
w8.pack()
 
w9 = Label(master, text = "Value min")
w9.pack()
w10 = Scale(master, from_=0, to=255, tickinterval=32, length=600, orient=HORIZONTAL, command=update_vmin)
w10.set(vmin)
w10.pack()
 
w11 = Label(master, text = "Value max")
w11.pack()
w12 = Scale(master, from_=0, to=255, tickinterval=32, length=600, orient=HORIZONTAL, command=update_vmax)
w12.set(vmax)
w12.pack()
 
mainloop()
 

We basically create 6 slider widgets (for the 3 ranges), and we attach a callback to each slider which will update the corresponding global variable and then send the corresponding updated range of values to JeVois.

• If you have not already done so, get a copy of the JeVois tutorials by opening a terminal and typing git clone https://github.com/jevois/jevois-tutorials.git
• Before you use the script, quit your serial terminal if you had started it, or you may get an error that the serial port is busy.
• Make sure the python TKinter and pySerial modules are installed, for example run sudo apt-get install python python-tk python-serial
• Make sure you are currently grabbing video in YUYV 320x254 @ 60.0 fps, so that the ObjectTracker module is loaded and running on JeVois.
• Then run the script:

    ~/jevois-tutorials/doc/snip/objectrackertuning.py 

  Note

  If you get some permission denied error, make sure you have permission to access the JeVois serial-over-USB device (see Connecting to JeVois using serial-over-USB: Linux host)

• You should see something like this:
  
• Go ahead and play with the sliders. Recommended approach is as follows:
  • Widen all 3 ranges to 0...255 (that is, set Hue min to 0, Hue max to 255, etc)
  • Point JeVois to a scene containing the object you are interested in, as well as a bunch of other objects that you are not interested in. Indeed, our goal will be to detect the objects of interest while rejecting (not detecting) other objects. The object should be big enough but not too big in the video captured by JeVois (another parameter in ObjectTracker called objectarea is by default set to look for objects of size between roughly 20x20 and 100x100 pixels). The ObjectTracker module draws green contours around all regions that are within the specified HSV range. Regions that also fit the area range are declared good detection and get an additional green circle drawn onto them.
  • Narrow down the Hue range (increase Hue min until just before you start losing the green contours around your object, then decrease Hue max likewise).
  • Then proceed with narrowing the Saturation and Value ranges.
  • Look at your objects of interest and at other things from various viewpoints, etc and fine tune the ranges.

Video of this tutorial in action

Where to go from here

You can further improve upon this tutorial in many ways. Here are some suggestions:

• Add widgets for the other parameters of the module, such as erodesize, dilatesize, etc
• Combine the min and max values into a fancier widget for a range, maybe a slider with 2 cursors, if such thing exists.
• Integrate the video capture into the python GUI, so that we do not have to start guvcview separately. Have a look here to get started. You will need to figure out how to handle video streaming in parallel with the GUI, as in this tutorial we end up just passing the control to the GUI's main loop. This pyimagesearch tutorial may be a good starting point.