JeVois Tutorials  1.21
JeVois Smart Embedded Machine Vision Tutorials
 
Share this page:
Loading...
Searching...
No Matches
Tuning the color-based object tracker using a python graphical interface

The color-based object tracker of JeVois is a simple but fast and effective way of tracking objects defined by a specific color. The behavior of the module can be fine-tuned by setting ranges for the hue (color tint), saturation (color fade), and value (color brightness) of the objects to track. While this can be achieved by hand, by using the approach studied in Changing machine vision parameters, quickly one becomes tired of typing many setpar commands while searching for appropriate settings of the 6 color-related parameters (hue min and max, saturation min and max, and value min and max).

Note
Note that since June 2018 it may be easier to use JeVois-A33: JeVois Inventor graphical user interface. The instructions below do not require JeVois Inventor are are provided in case you do not want to use the Inventor.

Here we write a simple graphical user interface in python to tune the parameters.

Hence, in this tutorial you will learn:

  • How to tune the ObjectTracker module to track any color you like.
  • How to communicate with JeVois from a python script.
  • How to develop a simple python GUI with sliders that update JeVois parameters when they are modified.

Getting started

  • This tutorial assumes a Linux host computer.
  • Connect JeVois to a host computer.
  • Start video capture software and select YUYV 320x254 @ 60.0 fps which should launch the ObjectTracker
  • Start a serial terminal and connect to the serial-over-USB port of JeVois.

Available parameters

Typing help in your serial terminal or inspecting the documentation of Object Tracker reveals several parameters. Here we will focus on the hrange, srange, and vrange parameters, which define the range of hue, saturation, and value that we want to track.

If you are not familiar with the HSV color space, check out this Wikipedia page.

Each of these three parameters is a range, that is, we can change them by specifying both the minimum and maximum values, separated by ..., for example, try:

setpar hrange 10...200

A more user-friendly way

We write a simple python script that creates some sliders that one can drag with the mouse. When the sliders are updated, we send the appropriate setpar command to JeVois. Consider this starter python script, which is located in ~/jevois-tutorials/doc/snip/objectrackertuning.py:

1#!/usr/bin/python
2
3# Needed packages: sudo apt-get install python python-tk python-serial
4
5# This tutorial is a simple program that allows one to adjust the hue, saturation, and value ranges of the ObjectTracker
6# module using sliders
7
8serdev = '/dev/ttyACM0' # serial device of JeVois
9
10from Tkinter import *
11import serial
12import time
13
14
15# default values for Hue, Saturation, and Value ranges:
16hmin = 95
17hmax = 110
18smin = 100
19smax = 255
20vmin = 60
21vmax = 253
22
23####################################################################################################
24# Send a command to JeVois and show response
25def send_command(cmd):
26 print "HOST>> " + cmd
27 ser.write(cmd + '\n')
28 out = ''
29 time.sleep(0.1)
30 while ser.inWaiting() > 0:
31 out += ser.read(1)
32 if out != '':
33 print "JEVOIS>> " + out, # the final comma suppresses extra newline, since JeVois already sends one
34
35####################################################################################################
36def update_hmin(val):
37 global hmin
38 global hmax
39 hmin = val
40 send_command('setpar hrange {0}...{1}'.format(hmin, hmax))
41
42####################################################################################################
43def update_hmax(val):
44 global hmin
45 global hmax
46 hmax = val
47 send_command('setpar hrange {0}...{1}'.format(hmin, hmax))
48
49####################################################################################################
50def update_smin(val):
51 global smin
52 global smax
53 smin = val
54 send_command('setpar srange {0}...{1}'.format(smin, smax))
55
56####################################################################################################
57def update_smax(val):
58 global smin
59 global smax
60 smax = val
61 send_command('setpar srange {0}...{1}'.format(smin, smax))
62
63####################################################################################################
64def update_vmin(val):
65 global vmin
66 global vmax
67 vmin = val
68 send_command('setpar vrange {0}...{1}'.format(vmin, vmax))
69
70####################################################################################################
71def update_vmax(val):
72 global vmin
73 global vmax
74 vmax = val
75 send_command('setpar vrange {0}...{1}'.format(vmin, vmax))
76
77####################################################################################################
78# Main code
79ser = serial.Serial(serdev, 115200, timeout=1)
80send_command('ping') # should return ALIVE
81
82master = Tk()
83
84w1 = Label(master, text = "Hue min")
85w1.pack()
86w2 = Scale(master, from_=0, to=255, tickinterval=32, length=600, orient=HORIZONTAL, command=update_hmin)
87w2.set(hmin)
88w2.pack()
89
90w3 = Label(master, text = "Hue max")
91w3.pack()
92w4 = Scale(master, from_=0, to=255, tickinterval=32, length=600, orient=HORIZONTAL, command=update_hmax)
93w4.set(hmax)
94w4.pack()
95
96w5 = Label(master, text = "Saturation min")
97w5.pack()
98w6 = Scale(master, from_=0, to=255, tickinterval=32, length=600, orient=HORIZONTAL, command=update_smin)
99w6.set(smin)
100w6.pack()
101
102w7 = Label(master, text = "Saturation max")
103w7.pack()
104w8 = Scale(master, from_=0, to=255, tickinterval=32, length=600, orient=HORIZONTAL, command=update_smax)
105w8.set(smax)
106w8.pack()
107
108w9 = Label(master, text = "Value min")
109w9.pack()
110w10 = Scale(master, from_=0, to=255, tickinterval=32, length=600, orient=HORIZONTAL, command=update_vmin)
111w10.set(vmin)
112w10.pack()
113
114w11 = Label(master, text = "Value max")
115w11.pack()
116w12 = Scale(master, from_=0, to=255, tickinterval=32, length=600, orient=HORIZONTAL, command=update_vmax)
117w12.set(vmax)
118w12.pack()
119
120mainloop()
121

We basically create 6 slider widgets (for the 3 ranges), and we attach a callback to each slider which will update the corresponding global variable and then send the corresponding updated range of values to JeVois.

  • If you have not already done so, get a copy of the JeVois tutorials by opening a terminal and typing git clone https://github.com/jevois/jevois-tutorials.git
  • Before you use the script, quit your serial terminal if you had started it, or you may get an error that the serial port is busy.
  • Make sure the python TKinter and pySerial modules are installed, for example run sudo apt-get install python python-tk python-serial
  • Make sure you are currently grabbing video in YUYV 320x254 @ 60.0 fps, so that the ObjectTracker module is loaded and running on JeVois.
  • Then run the script:
      ~/jevois-tutorials/doc/snip/objectrackertuning.py 
    Note
    If you get some permission denied error, make sure you have permission to access the JeVois serial-over-USB device (see Connecting to JeVois using serial-over-USB: Linux host)
  • You should see something like this:
  • Go ahead and play with the sliders. Recommended approach is as follows:
    • Widen all 3 ranges to 0...255 (that is, set Hue min to 0, Hue max to 255, etc)
    • Point JeVois to a scene containing the object you are interested in, as well as a bunch of other objects that you are not interested in. Indeed, our goal will be to detect the objects of interest while rejecting (not detecting) other objects. The object should be big enough but not too big in the video captured by JeVois (another parameter in ObjectTracker called objectarea is by default set to look for objects of size between roughly 20x20 and 100x100 pixels). The ObjectTracker module draws green contours around all regions that are within the specified HSV range. Regions that also fit the area range are declared good detection and get an additional green circle drawn onto them.
    • Narrow down the Hue range (increase Hue min until just before you start losing the green contours around your object, then decrease Hue max likewise).
    • Then proceed with narrowing the Saturation and Value ranges.
    • Look at your objects of interest and at other things from various viewpoints, etc and fine tune the ranges.

Video of this tutorial in action

Where to go from here

You can further improve upon this tutorial in many ways. Here are some suggestions:

  • Add widgets for the other parameters of the module, such as erodesize, dilatesize, etc
  • Combine the min and max values into a fancier widget for a range, maybe a slider with 2 cursors, if such thing exists.
  • Integrate the video capture into the python GUI, so that we do not have to start guvcview separately. Have a look here to get started. You will need to figure out how to handle video streaming in parallel with the GUI, as in this tutorial we end up just passing the control to the GUI's main loop. This pyimagesearch tutorial may be a good starting point.