JeVois Tutorials  1.9
JeVois Smart Embedded Machine Vision Tutorials
Share this page:
Hello JeVois

In this tutorial we create and compile a new JeVois machine vision module from scratch, and we run it on both the host computer (for testing) and on the JeVois platform hardware.

We will develop a module that just streams video from the camera sensor to the host computer, adding a message "Hello JeVois" on top of each video frame.

Preliminaries

Getting started

Writing the module

The skeleton code already provides an initial implementation for the HelloJeVois class, and for its process() function. It actually even already writes "Hello JeVois". So Let us just have a look at that code and it seems that we will not need to modify much (if anything) for this tutorial:

1 #include <jevois/Core/Module.H>
3 
4 // icon by Catalin Fertu in cinema at flaticon
5 
6 //! JeVois sample module
7 /*! This module is provided as an example of how to create a new standalone module.
8 
9  JeVois provides helper scripts and files to assist you in programming new modules, following two basic formats:
10 
11  - if you wish to only create a single module that will execute a specific function, or a collection of such modules
12  where there is no shared code between the modules (i.e., each module does things that do not relate to the other
13  modules), use the skeleton provided by this sample module. Here, all the code for the sample module is compiled
14  into a single shared object (.so) file that is loaded by the JeVois engine when the corresponding video output
15  format is selected by the host computer.
16 
17  - if you are planning to write a collection of modules with some shared algorithms among several of the modules, it
18  is better to first create machine vision Components that implement the algorithms that are shared among several of
19  your modules. You would then compile all your components into a first shared library (.so) file, and then compile
20  each module into its own shared object (.so) file that depends on and automatically loads your shared library file
21  when it is selected by the host computer. The jevoisbase library and collection of components and modules is an
22  example for how to achieve that, where libjevoisbase.so contains code for Saliency, ObjectRecognition, etc
23  components that are used in several modules, and each module's .so file contains only the code specific to that
24  module.
25 
26  @author Sample Author
27 
28  @videomapping YUYV 640 480 28.5 YUYV 640 480 28.5 SampleVendor HelloJeVois
29  @email sampleemail\@samplecompany.com
30  @address 123 First Street, Los Angeles, CA 90012
31  @copyright Copyright (C) 2017 by Sample Author
32  @mainurl http://samplecompany.com
33  @supporturl http://samplecompany.com/support
34  @otherurl http://samplecompany.com/about
35  @license GPL v3
36  @distribution Unrestricted
37  @restrictions None */
38 class HelloJeVois : public jevois::Module
39 {
40  public:
41  //! Default base class constructor ok
43 
44  //! Virtual destructor for safe inheritance
45  virtual ~HelloJeVois() { }
46 
47  //! Processing function
48  virtual void process(jevois::InputFrame && inframe, jevois::OutputFrame && outframe) override
49  {
50  // Wait for next available camera image:
51  jevois::RawImage const inimg = inframe.get(true);
52 
53  // We only support YUYV pixels in this example, any resolution:
54  inimg.require("input", inimg.width, inimg.height, V4L2_PIX_FMT_YUYV);
55 
56  // Wait for an image from our gadget driver into which we will put our results:
57  jevois::RawImage outimg = outframe.get();
58 
59  // Enforce that the input and output formats and image sizes match:
60  outimg.require("output", inimg.width, inimg.height, inimg.fmt);
61 
62  // Just copy the pixel data over:
63  memcpy(outimg.pixelsw<void>(), inimg.pixels<void>(), std::min(inimg.buf->length(), outimg.buf->length()));
64 
65  // Print a text message:
66  jevois::rawimage::writeText(outimg, "Hello JeVois!", 100, 230, jevois::yuyv::White, jevois::rawimage::Font20x38);
67 
68  // Let camera know we are done processing the input image:
69  inframe.done(); // NOTE: optional here, inframe destructor would call it anyway
70 
71  // Send the output image with our processing results to the host over USB:
72  outframe.send(); // NOTE: optional here, outframe destructor would call it anyway
73  }
74 };
75 
76 // Allow the module to be loaded as a shared object (.so) file:
friend friend class Module
void writeText(RawImage &img, std::string const &txt, int x, int y, unsigned int col, Font font=Font6x10)
unsigned int height
virtual ~HelloJeVois()
Virtual destructor for safe inheritance.
Definition: HelloJeVois.C:45
unsigned int fmt
std::shared_ptr< VideoBuf > buf
JEVOIS_REGISTER_MODULE(HelloJeVois)
virtual void process(jevois::InputFrame &&inframe, jevois::OutputFrame &&outframe) override
Processing function.
Definition: HelloJeVois.C:48
T const * pixels() const
unsigned int width
JeVois sample module.
Definition: HelloJeVois.C:38
void require(char const *info, unsigned int w, unsigned int h, unsigned int f) const

For fun, let us modify line 66 and add a couple more writeText() calls:

jevois::rawimage::writeText(outimg, "Hello JeVois!", 100, 230, jevois::yuyv::White, jevois::rawimage::Font20x38);
jevois::rawimage::writeText(outimg, "Hello JeVois!", 100, 30, jevois::yuyv::LightGreen, jevois::rawimage::Font20x38);
jevois::rawimage::writeText(outimg, "Hello JeVois!", 100, 400, jevois::yuyv::LightPink, jevois::rawimage::Font20x38);

Compiling and testing on host

Since we have already run rebuild-host.sh once and we have not added any new file since them (which would require that we run CMake again to detect them, for example by running rebuild-host.sh again), we can just use the partial compilation commands detailed in Programmer SDK and writing new modules (but note that, since we only have one file to compile here, this does not save us much time):

cd hbuild
make
sudo make install

Assuming that all of JeVois has been compiled previously, jevois-daemon should already exist and be ready to go. However, we need to add a new video mapping to /jevois/config/videomappings.cfg to let it know when and how to use our new module. We edit /jevois/config/videomappings.cfg (may require sudo if you get some permission denied error), and add a line:

YUYV 640 480 28.5 YUYV 640 480 28.5 Tutorial HelloJeVois

Or, with JeVois v1.3 and later, you can just type, in a Linux terminal:

sudo jevois-add-videomapping YUYV 640 480 28.5 YUYV 640 480 28.5 Tutorial HelloJeVois

Thus, the JeVois engine will now run module HelloJeVois from vendor Tutorial when video resolution 640x480 @ 28.5fps is requested. See Advanced topic: Video mappings and configuring machine vision modes for more info.

Now we need a regular webcam to test this, which should support YUYV 640x480 (frame rate not critical). Hopefully you have one. Of course we could use our JeVois camera configured in PassThrough mode to act like a regular dumb camera, but some people seem to be confused by this, so it may not be appropriate for this entry-level tutorial. If you do not have a regular USB webcam, just skip to the next section.

Start by just running

jevois-daemon

which will probably start in another mode (like DemoSaliency). Then, in the terminal in which you started jevois-daemon, type

listmappings

and note the mapping number for the one that we just added and that uses our HelloJevois module. For us it was mapping 16:

...
   16 - OUT: YUYV 640x480 @ 28.5fps CAM: YUYV 640x480 @ 28.5fps MOD: Tutorial:HelloJeVois
...

See Command-line interface user guide for more info about sending commands to JeVois and about the listmappings command.

So now just hit CTRL-C to stop jevois-daemon, and start it again using mapping 16 (or whatever that number was for you):

jevois-daemon --videomapping=16

You should see something like that in your terminal:

INF Engine::Engine: Loaded 44 vision processing modes.
ERR Engine::onParamChange: Cannot set cpu max frequency -- IGNORED
ERR Engine::onParamChange: Cannot set cpu frequency governor mode -- IGNORED
INF Engine::onParamChange: Using [stdio] hardware (4-pin connector) serial port
INF Engine::onParamChange: No USB serial port used
INF Engine::postInit: Starting camera device /dev/video0
ERR Engine::postInit: Could not access VFE turbo parameter -- IGNORED
INF Camera::Camera: [12] V4L2 camera /dev/video0 card JeVois-A33 Smart Camera bus usb-0000:09:00.0-1.1.4
INF Engine::postInit: Using display for video output
init done
INF Engine::setFormatInternal: OUT: YUYV 640x480 @ 28.5fps CAM: YUYV 640x480 @ 28.5fps MOD: Tutorial:HelloJeVois
INF Camera::setFormat: Camera set video format to 640x480 YUYV
INF Engine::setFormatInternal: Instantiating dynamic loader for /jevois/modules/Tutorial/HelloJeVois/HelloJeVois.so
INF Engine::setFormatInternal: Module [HelloJeVois] loaded, initialized, and ready.
INF Camera::streamOn: 6 buffers of 614400 bytes allocated
INF READY JEVOIS 1.0

and this in a window:

HelloJeVois1.png

Spicing it up a bit

Let us add a small title in the top-left corner, and also a display of frames/s, CPU frequency, and temperature at the bottom, like in the other JeVois demos. For this, we will use the jevois::Timer class.

Indeed, the JeVois framework provides a convenient jevois::Timer classes to help you measure how much time it takes to process each video frame. It works as follows:

The timer class will accumulate average statistics over 100 frames and will display those once in a while as information messages. In addition to writing messages to the console, stop() returns a string we can directly write onto our video frames. Timer does not report stats on every frame as this could slow us down too much, especially if sending those reports over serial port.

Let us first include the Timer declarations so we can use it:

and then we add it to our process() function, also adding the message at the top of the frame. The modified process() function now looks like this, with //////////////////////////////////////////////////////// markers to help you see the modifications (note that here we configured the timer to report every 60 frames, and to output text reports at the LOG_DEBUG level, which will not be printed out to console by default):

virtual void process(jevois::InputFrame && inframe, jevois::OutputFrame && outframe) override
{
static jevois::Timer timer("processing", 60, LOG_DEBUG); ////////////////////////////////////////////////////////
// Wait for next available camera image:
jevois::RawImage const inimg = inframe.get(true);
timer.start(); ////////////////////////////////////////////////////////
// We only support YUYV pixels in this example, any resolution:
inimg.require("input", inimg.width, inimg.height, V4L2_PIX_FMT_YUYV);
// Wait for an image from our gadget driver into which we will put our results:
jevois::RawImage outimg = outframe.get();
// Enforce that the input and output formats and image sizes match:
outimg.require("output", inimg.width, inimg.height, inimg.fmt);
// Just copy the pixel data over:
memcpy(outimg.pixelsw<void>(), inimg.pixels<void>(), std::min(inimg.buf->length(), outimg.buf->length()));
// Print a text message:
jevois::rawimage::writeText(outimg, "Hello JeVois!", 100, 230, jevois::yuyv::White, jevois::rawimage::Font20x38);
jevois::rawimage::writeText(outimg, "Hello JeVois!", 100, 30, jevois::yuyv::LightGreen, jevois::rawimage::Font20x38);
jevois::rawimage::writeText(outimg, "Hello JeVois!", 100, 400, jevois::yuyv::LightPink, jevois::rawimage::Font20x38);
// Let camera know we are done processing the input image:
inframe.done(); // NOTE: optional here, inframe destructor would call it anyway
////////////////////////////////////////////////////////
jevois::rawimage::writeText(outimg, "HelloJeVois Tutorial", 3, 3, jevois::yuyv::White);
// Show processing fps:
std::string const & fpscpu = timer.stop();
jevois::rawimage::writeText(outimg, fpscpu, 3, outimg.height - 13, jevois::yuyv::White);
////////////////////////////////////////////////////////
// Send the output image with our processing results to the host over USB:
outframe.send(); // NOTE: optional here, outframe destructor would call it anyway
}

To re-compile and re-install the module, we just type:

cd hbuild
make && sudo make install

And we can start jevois-daemon again and the output now looks like this (pointing the camera to something darker so we can see the small white text):

HelloJeVois2.png

This framerate is fast (this is a $15,000 host computer). We are now happy with our module. Let us run it inside the JeVois camera.

Cross-compiling for JeVois hardware

Since we have already run rebuild-platform.sh on our module and we have not added any new file since then, we can just do a partial re-compilation for platform:

cd ~/hellojevois/pbuild
make && make install
Note
If you get an error like cc1plus: fatal error: jevois/Config/Config.H: No such file or directory, you just need to run rebuild-platform.sh in your jevois directory first.

JeVois provides three basic ways of installing new modules to microSD:

We look at these methods below.

Defining a video mapping for our module

One last thing before we install to the camera is that we will include with our module an additional video mapping that we want to be automatically added to the videomappings.cfg of our JeVois camera as the module gets installed. For this, we edit ~/hellojevois/src/Module/HelloJeVois/postinstall as follows:

#!/bin/sh
# This script is executed once after the module is installed by JeVois if it was added to the jevois/packages/ directory
# of the microSD card as a .jvpkg file. The script is deleted from the microSD card after execution.
#
# The caller script will set the current directory to the location of this script before launching the script.
# Add our video mappings to the main mappings file:
jevois-add-videomapping YUYV 640 480 28.5 YUYV 640 480 28.5 Tutorial HelloJeVois
# Example of a simple message:
echo "HelloJeVois is now installed"

When the module is installed onto the smart camera, this mapping will automatically be added to the list of existing mappings. This will be achieved by JeVois automatically running the postinstall script that is in the directory of our module. If such script exists, it will be run just after the package is unpacked. It will be run with the current directory set to the directory of the module. By default, postinstall contains simple commands that just append the videomappings.cfg of the new module to the main videomappings.cfg of JeVois, but you can edit this script if you need additional custom configurations to be run as your package is installed.

Installing to live JeVois camera

With JeVois v1.3 and later, you do not need to manually take out or even manually export the microSD inside JeVois anymore. A new build option --live can do it automatically. Just connect JeVois and allow it to start, then type:

./rebuild-platform.sh --live

which will:

Note
Once you select a destination in rebuild-platform.sh (none, staging, microsd, or live), that will remain for subsequent make install commands, until you change it by re-running rebuild-platform.sh with another destination.

Installing to JeVois microSD card

Insert a properly installed JeVois microSD (follow the steps at JeVois Start to make one from a disk image) into your computer, either physically inserted into your host computer using a microSD-to-USB adapter, or exported live by JeVois using the usbsd command in the JeVois command-line interface; see Command-line interface user guide).

The microSD card must be be available under /media/username/JEVOIS/ where username is your Linux user name.

Once the microSD is available to the host, run:

cd ~/hellojevois
./rebuild-platform.sh --microsd

Modules, config files, etc will be installed to the live microSD card at /media/username/JEVOIS/ where username is your Linux user name.

Note
Once you select a destination in rebuild-platform.sh (none, staging, microsd, or live), that will remain for subsequent make install commands, until you change it by re-running rebuild-platform.sh with another destination.

Here is a simple command to export the microSD of a live JeVois camera without having to start a terminal program, etc (make sure JeVois is not streaming video):

sudo sh -c 'echo usbsd > /dev/ttyACM0'

or, with JeVois v1.2 or later: jevois-usbsd start

If you have a postinstall script in your module that adds new video mappings, those will get appended to the main videomappings.cfg next time JeVois reboots.

Installing to JeVois hardware using a .jvpkg package

Let us create a package with all the files of our module:

cd ~/hellojevois/pbuild
make jvpkg

which creates ~/hellojevois/Tutorial_hellojevois.jvpkg that contains everything needed to install this module onto our smart camera.

We connect the microSD card of our smart camera to our host computer and we copy the package file to JEVOIS:/packages/ on the microSD card. Next time we start up the smart camera, it will unpack the package, install the module .so file, run the postinstall script which will update the video mappings, etc and we will be ready to go. Something like this:

cp Tutorial_hellojevois.jvpkg /media/itti/JEVOIS/packages/

(replace itti above by your username). Then insert the microSD into our JeVois smart camera and start it up.

Note
With JeVois v1.1 and later, you do not need to physically remove the microSD from JeVois, and you can instead just export it as a virtual USB flash drive while JeVois is connected to your host computer and running:

With JeVois v1.2 and later, you can just type jevois-usbsd start to start exporting th emicroSD, and jevois-usbsd stop to eject it and restart JeVois.

Start guvcview on your host computer. Note that JeVois may take a bit more time to boot up as it is unpacking and installing our module. This is will be done this time only, and the package will be deleted from microSD once all its files have been installed.

Set guvcview to YUYV 640x480 and you will notice the new 28.5fps framerate (will show up as 57/2 in some versions of guvcview). Select it and here you are:

HelloJeVois3.png

If things go wrong here (the JeVois smart camera fails to start after you copied that .jvpkg file to JEVOIS:/packages/, or the 640x480 @ 28.5fps mode does not appear in guvcview), check the following:

Installing to JeVois hardware using the staging area of jevois-sdk

Use this method if you want to reflash the entire microSD using the latest operating system, core files, etc provided by jevois-sdk.

The entire operating system for JeVois is created under the jevois-sdk and the resulting files are placed into two staging directories: /var/lib/jevois-build for system files, and /var/lib/jevois-microsd for files that will go into the JEVOIS partition. See Organization of JeVois files on host and platform for details. It can then be installed onto a fresh new microSD card.

To include your new module into the staging area, run this:

cd ~/hellojevois
./rebuild-platform.sh --staging

which will copy all the files to the staging area.

If you have a postinstall script in your module that adds new video mappings, those will still get appended to the main videomappings.cfg next time JeVois reboots.

Then you can flash a new microSD card as usual (see Flashing to microSD card):

sudo jevois-flash-card -y /dev/sdX

(replace sdX above by the device for your microSD).

Warning
Once you compile with the --staging option, the files will be copied to the staging area and will never be deleted. So use with caution so that you do not get confused later on because an old version of a module was installed to staging while you are trying to install a new one as a package, etc. To figure out where the files are in staging, run find /var/lib/jevois-microsd -name HelloJevois.so and you can then delete these files manually from staging (and the whole modules/Tutorial directory).