Simple image filtering demo using 4-core CPU processing and OpenGL-ES 2.0 shaders on the Mali-400MP2 GPU.
By Laurent Ittiitti@usc.eduhttp://jevois.orgGPL v3
 Language:   C++            Supports mappings with USB output:   Yes            Supports mappings with NO USB output:   No
 Video Mapping:   GREY 160 495 60.0 YUYV 160 120 60.0 JeVois DemoCPUGPU

Module Documentation

In this demo, we compute saliency and gist over our 4 CPU cores while we also compute 4 different image filters over the GPU, finally combining all results into a single grayscale image:

  • saliency: multicore CPU-based detection of the most conspicuous (most attention-grabbing) object in the field of view.
  • GPU filter 1: Sobel edge detector
  • GPU filter 2: Median filter
  • GPU filter 3: Morphological erosion filter
  • GPU filter 4: Morphological dilation filter

For an introduction to visual saliency, see

Video output

The video output is arranged vertically, with, from top to bottom:

  • Sobel results (same size as input image)
  • Median filter results (same size as input image)
  • Morphological erosion filter results (same size as input image)
  • Morphological dilation filter results (same size as input image)
  • Saliency results: from left to right: saliency map, color map, intensity map, orientation map, flicker map, motion map. Map size is input size divided by 8 horizontally and vertically. Gist vector is appended to the right but note that at 160x120 it is truncated (see source code for details).

Serial Messages

This module can send standardized serial messages as described in Standardized serial messages formatting, where all coordinates and sizes are standardized using Helper functions to convert coordinates from camera resolution to standardized. One message is issued on every video frame at the temporally filtered attended (most salient) location:

  • Serial message type: 2D
  • id: always sm (shorthand for saliency map)
  • x, y: standardized 2D coordinates of temporally-filtered most salient point
  • w, h: always 0, 0
  • extra: none (empty string)
ParameterTypeDescriptionDefaultValid Values
(Saliency) cweightbyteColor channel weight255-
(Saliency) iweightbyteIntensity channel weight255-
(Saliency) oweightbyteOrientation channel weight255-
(Saliency) fweightbyteFlicker channel weight255-
(Saliency) mweightbyteMotion channel weight255-
(Saliency) centerminsize_tLowest (finest) of the 3 center scales2-
(Saliency) deltaminsize_tLowest (finest) of the 2 center-surround delta scales3-
(Saliency) smscalesize_tScale of the saliency map4-
(Saliency) mthreshbyteMotion threshold0-
(Saliency) fthreshbyteFlicker threshold0-
(Saliency) msflickboolUse multiscale flicker computationfalse-
(Kalman2D) usevelboolUse velocity tracking, in addition to positionfalse-
(Kalman2D) procnoisefloatProcess noise standard deviation0.003F-
(Kalman2D) measnoisefloatMeasurement noise standard deviation0.05F-
(Kalman2D) postnoisefloatA posteriori error estimate standard deviation0.3F-
Detailed docs:DemoCPUGPU
Copyright:Copyright (C) 2016 by Laurent Itti, iLab and the University of Southern California
License:GPL v3
Support URL:
Other URL:
Address:University of Southern California, HNB-07A, 3641 Watt Way, Los Angeles, CA 90089-2520, USA