JeVois  1.10
JeVois Smart Embedded Machine Vision Toolkit
Share this page:
JeVois Inventor graphical user interface

JeVois Inventor is a graphical frontend for JeVois Smart Cameras. It makes programming new computer vision pipelines that run on JeVois very easy, using full Python 3.6 + OpenCV 3.4.

This frontend was created for people who do not read user manuals, so feel free to explore around!

Nevertheless, here are a few tips.

Note
JeVois Inventor is currently in beta. Please help us refine this software by emailing comments and bug reports to jevoi.nosp@m.s.or.nosp@m.g@gma.nosp@m.il.c.nosp@m.om

Installing JeVois Inventor

  • Windows:
    • Tested on Windows 7 and Windows 10.
    • Download jevois-inventor_XXX.zip from http://jevois.org/start (XXX will vary depending on version)
    • Windows 7 (not 8 or 10) users need to install a driver, see http://jevois.org/doc/USBserialWindows.html
    • Extract zip contents
    • Double-click on jevois-inventor.exe
    • If the JeVois camera is not detected, see below, under "Troubleshooting"
  • Linux:
    • Available and tested for 18.04 amd64 (bionic) and 16.04 amd64 (xenial)
    • Download jevois-inventor_XXX.deb from http://jevois.org/start (XXX will vary depending on version)
    • sudo dpkg -i jevois-inventor_XXX.deb # (XXX will vary depending on version)
    • sudo killall ModemManager # (ModemManager interferes with JeVois)
    • jevois-inventor
    • If you want to avoid having to change permissions for the video and serial ports of JeVois each time you connect it, you need to add your user to the video and dialout groups, as follows:
          sudo apt purge modemmanager
          sudo usermod -aG dialout $USER
          sudo usermod -aG video $USER
      and reboot your computer for the settings to take effect.
  • MacOS:
    • Tested on High Sierra. Does not work on Snow Leopard, which is too old. Sorry we do not have versions in between to test...
    • Download jevois-inventor_XXX.dmg from http://jevois.org/start (XXX will vary depending on version)
    • Double-click on the DMG file to open it
    • Drag jevois-inventor to your desktop or Applications folder
    • Double-click on jevois-inventor. If your mac refuses to open it, CTRL+click on the JeVois Inventor icon and select Open from the pull-down menu. This should now open it.

Your JeVois Smart Camera

See also:

Detecting your JeVois Smart Camera

Detection is fully automatic. No device to specify, no port to look for.

Just start JeVois Inventor, insert a microSD card with the latest JeVois software into your camera, and connect the camera to your host computer.

As soon as the camera is detected, the DemoSaliency machine vision module will start automatically.

You can disconnect JeVois at any time, and JeVois Inventor should detect that. Next time you connect JeVois, JeVois Inventor should also detect it again.

Troubleshooting:

Trying out various machine vision modules

Use the Vision Module menu to select a given machine vision module.

Modules written in Python are highlighted in bold font. You can edit the code for these modules directly in JeVois Inventor (see below).

Under the Info tab you can browse the module's documentation. You can zoom in/out by pressing CTRL + mouse wheel.

Note
Currently, only YUYV capture is supported by JeVois Inventor, which we believe is a limitation of the QCamera class of Qt (the widget toolkit we used to develop JeVois Inventor). We are looking into fixing this, likely by fixing Qt at its core.

For more information: Now may be a good time to familiarize yourself with a few key concepts: Concepts used throughout this documentation

Adjusting parameters

Many machine vision modules expose parameters that can be tuned by the user.

Click on the Parameters tab to see the current module's parameter.

  • Hover over a parameter's widget to see the description of what this parameter does
  • Hover over a parameter's name to see its full machine-oriented name and data type
  • Click the Reset button to reset a parameter to its default value
  • Toggle Show Frozen Parameters to show parameters which cannot be changed at runtime (those usually can only be set once, when the module is loaded, through config files).
  • Toggle Show System Parameters to show parameters which relate to the JeVois core system, as opposed to the current machine vision module.

Communicating with JeVois through its command-line console

JeVois provides a simple command-line interface to allow one to run commands, set parameters, etc over the 4-pin serial port of JeVois, or over the Serial-over-USB port (which JeVois Inventor uses).

Mainly, this interface is intended to allow micro-controllers or host computers to configure JeVois. In fact, each time you click a button in JeVois Inventor, a command is sent to JeVois using this command-line interface.

  • Click on the input line at the bottom to enter a command. The command input line has full history, so you can use the up/down cursor keys to re-enter previously entered commands.
  • Hover your mouse over the ? next to the Enter button to see a list of currently available commands.
  • At the top left, choose where to send log messages (nowhere, to USB, to 4-pin serial port, or both). Log messages are emitted by some modules to let you know details of how they are running, or to let you know about any error condition.
  • At the top right, choose where to send module output messages (nowhere, to USB, to 4-pin serial port, or both). Module output messages contain results of what the current machine vision pipeline has found in the input video. For example, it may stream the names of recognized objects, or the coordinates and sizes of detected marker tags or color blobs.

In the example above, JeVois has detected ArUco marker tags (with IDs U42, U27, U18, and U43), two blue color blobs (detected by color detector named blob0), and one yellow blob (detected by blob1). For each detected object, JeVois also sends the 4 normalized (between -1000 and 1000, with 0 at the center) 2D coordinates of their 4 corners (Detail message format).

For more information about these messages and about their level of detail (Terse, Normal, Detail, or Fine), see Standardized serial messages formatting

For more information about the JeVois command-line interface, see Command-line interface user guide

Changing camera settings

It is sometimes useful to play with low-level camera sensor controls, such as white balance, gain, and exposure, to optimize the performance of a machine vision algorithm. Select the Camera tab to access these sensor controls.

  • Note how some controls are dependent upon others (e.g., gain is disabled and cannot be changed when autogain is active).
  • Click Reset next to each control to revert to its default value.
  • Special note about absolute exposure control: To enable absolute exposure, select manual for the autoexp parameter first. This will activate the absexp and gain sliders and will also turn off autogain. Note that the usable range for absexp depends on current camera sensor resolution and frame rate, as we cannot expose the image for longer than roughly one frame period (inverse of the frame rate). So it is normal that, in most cases, when you drag the slider all the way to the maximum, it will snap back to the allowed maximum for the current video resolution and frame rate.

Config files

Config files allow one to set some parameters for JeVois or for a given module, to be used every time JeVois starts or every time that module is invoked. You can think of this as setting default preferences for JeVois or for a given module.

Click on the Config tab, and then you can select one of 4 config files:

  • JeVois initscript.cfg: a global config file that is run once when JeVois starts up. This file is useful if you want JeVois to automatically launch one particular machine vision module without any user intervention (headless mode), for example when JeVois is used in a robot that only has an Arduino controller an no host computer to run JeVois Inventor.
  • JeVois videomappings.cfg: This is a list of mappings between video resolutions streamed by JeVois over USB and the corresponding camera sensor resolutions, and which machine vision module to run. This file is how one decides which machine vision module will run on JeVois when a host computer requests a given video format from JeVois.

    For more information, see User guide to video modes and mappings

  • Module's params.cfg: This contains parameters for the current module that should be set each time that module is loaded, and before it is started. For example, the deep neural network modules like TensorFlowEasy use this file to configure which neural network will be used.
  • Module's script.cfg: This is a script file for the current module that is run after the module is loaded and initialized. One can put additional parameter or camera control settings in this file. For example, ArUcoBlob uses this file to configure the four colors of its four color blob detectors that run in parallel.

Note the two middle buttons at the bottom:

  • C: paste current camera controls (as were adjusted by hand using the Camera tab) to the currently shown file (only script files can accept this)
  • P: paste the current module parameters (as were adjusted by hand using the Parameters tab) to the currently shown file (both params and script files can accept this)

Editing machine vision code

Note
Currently, this is only supported for Python modules. It is unclear whether we will be able to support this for C++ modules as well. So you should select a Python module (in bold in the Vision Module pull-down menu). Source code for C++ modules can be seen, but currently is read-only in JeVois Inventor.

Select the Code tab to access an editor to the source code of the currently running module.

Edit the code, press CTRL-S to save to JeVois, and you will see results immediately.

Don't be shy, just try some OpenCV code and see what happens. The basic idea is that you are given an image inimg from the camera sensor, and your task is to create an output image outimg that will be sent to the host computer over USB.

A few notes about the editor:

  • it features syntax highlighting (different colors for keywords, comments, etc)
  • automatic indent (indent increases if you press RETURN after a :)
  • use TAB to add indent, and Shift+TAB to reduce indent
  • indent (leading spaces) shown with gray dots

Creating a new machine vision module

Select New Python Module... from the Vision Module pull-down menu.

Enter the data. All fields are required. The Finish button will not be enabled until you have entered acceptable data. Hover your mouse over each field to see tips.

For simple modules, you would typically select the same resolution for USB output and camera sensor. For more complex modules, for example showing a side-by-side output with the original video frame at left and the machine vision results at right, they could differ (e.g., in the side-by-side example, camera sensor resolution might be 320x240 and USB output would then be 640x240). It is the responsibility of the module writer to create the output image with correct resolution, which the JeVois core will then send over USB to the host computer.

To get started with writing your new modules, check out Tutorial on how to write new machine vision modules in Python, Programming machine vision modules with Python, and Programmer tutorials

System settings

If you want to stop the video stream, use the System tab. This is needed before you start a module in headless mode using the setmapping2 command. See, for example, the tutorial on making a pan/tilt head for JeVois, section on Running with no USB output.

You can also export the microSD inside JeVois as a virtual flash drive to your host computer. Once enabled, a new flash drive should appear on your desktop and you can browse the files on the microSD of JeVois. Very useful to backup your custom modules, or to load your newly trained deep neural networks.

While you are browsing the microSD inside JeVois, you cannot change or use machine vision modules. When you are done, properly eject the virtual USB drive from your host (safely remove, drag to trash, eject button, etc) and JeVois will restart to take into account the changes you have made to its microSD card.

Going deeper

JeVois Inventor is just a simple front-end to the core of JeVois. You can do more with JeVois than what JeVois Inventor supports. See http://jevois.org/doc/ for more comprehensive documentation.

In particular, check out:

Release notes

JeVois Inventor 0.1.0

  • Initial release

JeVois Inventor 0.2.0

  • Better device detection on Windows platforms
  • Attached spinboxes to all sliders
  • Added line numbers and highlighting of current line to all editors
  • Added links to various doc pages from File menu.

Troubleshooting for Windows

JeVois Inventor tries to detect the JeVois camera automatically, so that you do not have to select its device each time you start the inventor or restart the camera.

The JeVois Smart Camera is a composite USB device with three sub-devices:

  • a USB Video Class (UVC) video camera, used to stream video over USB;
  • a CDC-ACM Serial-over-USB port, used to send commands to JeVois;
  • a mass-storage device, used to access the microSD card inside JeVois as a virtual flash drive.

To work, JeVois Inventor needs to detect at least the first two sub-devices. On Linux and MacOS, this works well by enumerating all connected cameras and serial ports, and finding the one that has the word JeVois in its Manufacturer or Product descriptions.

However, this functionality (provided by the Qt toolkit of widgets which JeVois Inventor is programmed with) does not seem to work well on Windows platforms. Even though the JeVois camera sends manufacturer and product descriptions to the host computer when it is connected, those just seem to be ignored by Windows. Windows instead assigns names to JeVois that appear to be whatever names are present in driver (.inf) files previously loaded on your computer.

Try the following if your JeVois camera is not detected by the Inventor:

  • Disconnect the camera and open your Control Panel and then Device Manager.
  • Expand the entries for Imaging devices (or Cameras on Windows 10), and for Ports (COM & LPT).
  • Connect JeVois and wait for Windows to detect it.
  • One new imaging device and one new port should appear; write down their names. In the example below, JeVois was assigned the name UVC Camera for the camera sub-device, and Gadget Serial for the serial-over-USB sub-device. On Windows 7, you will likely need to use jevois-serial.inf as explained in Connecting to JeVois using serial-over-USB: Windows host to install the correct USB-serial driver. Depending on your Windows version and history of installed drivers, these names will vary widely. They will correctly contain the word JeVois if you have not previously used USB cameras or serial-over-USB adapters on your computer (i.e., JeVois is the first such device detected, and Windows will then fetch and use the correct name advertized by the device).

    jvinv-devices.png
  • Starting with JeVois Inventor 0.2.0, if JeVois is not automatically detected, you will be invited to manually select the camera, or serial-over-USB, or both sub-devices. Use the names you noted in the Device Manager when you first connected your JeVois camera:

    jvinv-devsel.png
  • To avoid the need to select device names each time you restart the Inventor, you can try to uninstall existing drivers, so that the correct name will be pulled next time Windows detects JeVois and re-installs the drivers:
    • Connect JeVois and wait for Windows to detect it.
    • In the Device Manager, right-click on the device and select Uninstall. Accept to uninstall. Do this both for the camera device and (Windows 7 only) serial port device (you cannot uninstall the Microsoft serial port device on Windows 10).
    • Disconnect JeVois.
    • After a minute, connect JeVois again. You should see some messages about a new device being detected. Do not touch anything and let Windows install the drivers. After a minute or two, the drivers should have installed and be ready to use.
    • For the serial-over-USB port, on Windows 10 it is built-in and you cannot change it, its name is fixed to "USB Serial Device". For this reason, JeVois Inventor also accepts this name in addition to a name containing the word JeVois. On Windows 7, use jevois-serial.inf as explained in Connecting to JeVois using serial-over-USB: Windows host to install the correct driver.
    • Under Imaging Devices (or Cameras) in the Device Manager, confirm that the name now is "JeVois-A33 Video Camera" for the camera sub-device of JeVois.
    • Under Ports (COM & LPT), the name should now be "JeVois-A33 Serial-over-USB" (Windows 7), or "USB Serial Port" (Windows 10).
    • The inventor should now detect your camera automatically with no need for manual device selection.
jvinv-gooddev.png
Above: Windows 7 devices after un-installing camera driver and updating serial driver
jvinv-gooddev10.png
Above: Windows 10 devices after un-installing camera driver (serial driver cannot be changed)