This module computes saliency and gist over our 4 CPU cores while also computing 4 different image filters over the GPU, finally combining all results into a single grayscale image:

• saliency: multicore CPU-based detection of the most conspicuous (most attention-grabbing) object in the field of view.
• GPU filter 1: Sobel edge detector
• GPU filter 2: Median filter
• GPU filter 3: Morphological erosion filter
• GPU filter 4: Morphological dilation filter

For an introduction to visual saliency, see http://ilab.usc.edu/bu/

Also see DemoSaliency, JeVoisIntro, DarknetSaliency for more about saliency.

Video output

The video output is arranged vertically, with, from top to bottom:

• Sobel filter results (same size as input image)
• Median filter results (same size as input image)
• Morphological erosion filter results (same size as input image)
• Morphological dilation filter results (same size as input image)
• Saliency results (those are very small): from left to right: saliency map, color map, intensity map, orientation map, flicker map, motion map. Map size is input size divided by 8 horizontally and vertically. Gist vector is appended to the right but note that at 160x120 it is truncated (see source code for details).

Serial Messages

This module can send standardized serial messages as described in Standardized serial messages formatting, where all coordinates and sizes are standardized using Helper functions to convert coordinates from camera resolution to standardized. One message is issued on every video frame at the temporally filtered attended (most salient) location:

• Serial message type: 2D
• id: always sm (shorthand for saliency map)
• x, y: standardized 2D coordinates of temporally-filtered most salient point
• w, h: always 0, 0
• extra: none (empty string)

See Standardized serial messages formatting for more on standardized serial messages, and Helper functions to convert coordinates from camera resolution to standardized for more info on standardized coordinates.