JeVoisBase  1.0
JeVois Smart Embedded Machine Vision Toolkit Base Modules
Saliency Class Reference

Simple wrapper class around Rob Peter's C-optimized, fixed-point-math visual saliency code. More...

#include </lab/itti/jevois/software/jevoisbase/src/Components/Saliency/Saliency.H>

Inheritance diagram for Saliency:
Collaboration diagram for Saliency:

Classes

struct  visitor_data
 

Public Member Functions

 Saliency (std::string const &instance)
 Constructor. More...
 
virtual ~Saliency ()
 Destructor. More...
 
void process (jevois::RawImage const &input, bool do_gist)
 Process a raw YUYV image. Results are stored in the Saliency class. More...
 
void process (cv::Mat const &input, bool do_gist)
 Process an RGB image. Results are stored in the Saliency class. More...
 
void waitUntilDoneWithInput () const
 Wait until process() is done using the input image. More...
 
void getSaliencyMax (int &x, int &y, intg32 &value)
 Get location and value of max point in the saliency map. More...
 
void inhibitionOfReturn (int const x, int const y, float const sigma)
 Inhibit the saliency map around a point, sigma is in pixels at the sacle of the map. More...
 
- Public Member Functions inherited from jevois::Component
 Component (std::string const &instance)
 
virtual ~Component ()
 
std::shared_ptr< ComponentgetSubComponent (std::string const &instance) const
 
std::shared_ptr< Comp > addSubComponent (std::string const &instance, Args &&...args)
 
void removeSubComponent (std::shared_ptr< Comp > &component)
 
void removeSubComponent (std::string const &instance, bool warnIfNotFound=true)
 
std::shared_ptr< Comp > getSubComponent (std::string const &instance) const
 
bool isTopLevel () const
 
bool initialized () const
 
std::string const & className () const
 
std::string const & instanceName () const
 
std::vector< std::string > setParamVal (std::string const &paramdescriptor, T const &val)
 
void setParamValUnique (std::string const &paramdescriptor, T const &val)
 
std::vector< std::pair< std::string, T > > getParamVal (std::string const &paramdescriptor) const
 
getParamValUnique (std::string const &paramdescriptor) const
 
std::vector< std::string > setParamString (std::string const &paramdescriptor, std::string const &val)
 
void setParamStringUnique (std::string const &paramdescriptor, std::string const &val)
 
std::vector< std::pair< std::string, std::string > > getParamString (std::string const &paramdescriptor) const
 
std::string getParamStringUnique (std::string const &paramdescriptor) const
 
void freezeParam (std::string const &paramdescriptor)
 
void unFreezeParam (std::string const &paramdescriptor)
 
void freezeAllParams ()
 
void unFreezeAllParams ()
 
std::string descriptor () const
 
void setParamsFromFile (std::string const &filename)
 
std::istream & setParamsFromStream (std::istream &is, std::string const &absfile)
 
void setPath (std::string const &path)
 
std::string absolutePath (std::string const &path="")
 
std::shared_ptr< Comp > addSubComponent (std::string const &instance, Args &&...args)
 
void removeSubComponent (std::shared_ptr< Comp > &component)
 
void removeSubComponent (std::string const &instance, bool warnIfNotFound=true)
 
std::shared_ptr< Comp > getSubComponent (std::string const &instance) const
 
bool isTopLevel () const
 
bool initialized () const
 
std::string const & className () const
 
std::string const & instanceName () const
 
std::vector< std::string > setParamVal (std::string const &paramdescriptor, T const &val)
 
void setParamValUnique (std::string const &paramdescriptor, T const &val)
 
std::vector< std::pair< std::string, T > > getParamVal (std::string const &paramdescriptor) const
 
getParamValUnique (std::string const &paramdescriptor) const
 
std::vector< std::string > setParamString (std::string const &paramdescriptor, std::string const &val)
 
void setParamStringUnique (std::string const &paramdescriptor, std::string const &val)
 
std::vector< std::pair< std::string, std::string > > getParamString (std::string const &paramdescriptor) const
 
std::string getParamStringUnique (std::string const &paramdescriptor) const
 
void freezeParam (std::string const &paramdescriptor)
 
void unFreezeParam (std::string const &paramdescriptor)
 
void freezeAllParams ()
 
void unFreezeAllParams ()
 
std::string descriptor () const
 
void setParamsFromFile (std::string const &filename)
 
std::istream & setParamsFromStream (std::istream &is, std::string const &absfile)
 
void setPath (std::string const &path)
 
std::string absolutePath (std::string const &path="")
 
- Public Member Functions inherited from jevois::ParameterRegistry
virtual ~ParameterRegistry ()
 

Public Attributes

struct env_image salmap
 The saliency map. More...
 
struct env_image intens
 
struct env_image color
 
struct env_image ori
 
struct env_image flicker
 
struct env_image motion
 
unsigned char * gist
 Gist vector has 1152 entries, 72 feature maps * 16 values/map. More...
 
size_t const gist_size
 

Related Functions

(Note that these are not member functions.)

#define JEVOIS_DECLARE_PARAMETER(ParamName, ParamType, ...)
 Parameter. More...
 
#define JEVOIS_DECLARE_PARAMETER(ParamName, ParamType, ...)
 Parameter. More...
 
#define JEVOIS_DECLARE_PARAMETER(ParamName, ParamType, ...)
 Parameter. More...
 
#define JEVOIS_DECLARE_PARAMETER(ParamName, ParamType, ...)
 Parameter. More...
 
#define JEVOIS_DECLARE_PARAMETER(ParamName, ParamType, ...)
 Parameter. More...
 
#define JEVOIS_DECLARE_PARAMETER(ParamName, ParamType, ...)
 Parameter. More...
 
#define JEVOIS_DECLARE_PARAMETER(ParamName, ParamType, ...)
 Parameter. More...
 
#define JEVOIS_DECLARE_PARAMETER(ParamName, ParamType, ...)
 Parameter. More...
 
#define JEVOIS_DECLARE_PARAMETER(ParamName, ParamType, ...)
 Parameter. More...
 
#define JEVOIS_DECLARE_PARAMETER(ParamName, ParamType, ...)
 Parameter. More...
 
#define JEVOIS_DECLARE_PARAMETER(ParamName, ParamType, ...)
 Parameter. More...
 
void drawMap (jevois::RawImage &img, env_image const *fmap, unsigned int xoff, unsigned int yoff, unsigned int scale)
 Draw a saliency map or feature map in a YUYV image. More...
 
void drawMap (jevois::RawImage &img, env_image const *fmap, unsigned int xoff, unsigned int yoff, unsigned int scale, unsigned int bitshift)
 Draw a saliency map or feature map in a YUYV image, applying some right bitshift to values. More...
 
void drawGist (jevois::RawImage &img, unsigned char const *gist, size_t gistsize, unsigned int xoff, unsigned int yoff, unsigned int width, unsigned int scale)
 Draw a gist vector in a YUYV image as a rectangle of width width*scale and correct height. More...
 

Additional Inherited Members

- Protected Member Functions inherited from jevois::Component
virtual void preInit ()
 
virtual void postInit ()
 
virtual void preUninit ()
 
virtual void postUninit ()
 
virtual void preInit ()
 
virtual void postInit ()
 
virtual void preUninit ()
 
virtual void postUninit ()
 
- Protected Member Functions inherited from jevois::ParameterRegistry
void addParameter (ParameterBase *const param)
 
void removeParameter (ParameterBase *const param)
 
void callbackInitCall ()
 

Detailed Description

Simple wrapper class around Rob Peter's C-optimized, fixed-point-math visual saliency code.

This component implements a visual saliency detector using the algorithm from Itti et al., IEEE Trans PAMI,

  1. The saliency map highlights locations in an video stream that stand out and would attract the attention of a human. In addition, this component computes the gist of the scene, a low-dimentional statistical summary of the whole scene that can be used for scene classification.

Note that the Saliency component here imposes some restrictions on the saliency computations:

  • we enforce that always 6 feature maps are computed for each channel, which is obtained by using 3 center scales and 2 center-surround scale deltas. In this component, only the finest center scale and smallest delta can be specified, as well as the saliency map scale. This is so that the gist vector can have a fixed size.
  • number of orientations in the orientation channel is fixed at 4, number of directions in the motion channel is fixed at 4. This is again to obtain a fixed gist vector size.
  • we always consider all of C, I O, F and M channels as opposed to having a more dynamic collection of channels as done in other implementations of this model (see, e.g., http://iLab.usc.edu/toolkit/). This is again so that we have fixed gist size and available output maps. Note that some channels will not be computed if their weight is set to zero, and instead the maps will be empty and the gist entries will be zeroed out.

See the research paper at http://ilab.usc.edu/publications/doc/Itti_etal98pami.pdf

Definition at line 95 of file Saliency.H.

Constructor & Destructor Documentation

Saliency::~Saliency ( )
virtual

Destructor.

Definition at line 106 of file Saliency.C.

References env_img_make_empty(), env_motion_channel_destroy(), env_pyr_make_empty(), and gist.

Member Function Documentation

void Saliency::getSaliencyMax ( int &  x,
int &  y,
intg32 value 
)

Get location and value of max point in the saliency map.

Definition at line 672 of file Saliency.C.

References env_image::dims, env_dims::h, LFATAL, env_image::pixels, salmap, and env_dims::w.

void Saliency::inhibitionOfReturn ( int const  x,
int const  y,
float const  sigma 
)

Inhibit the saliency map around a point, sigma is in pixels at the sacle of the map.

Definition at line 686 of file Saliency.C.

References env_image::dims, env_dims::h, LFATAL, env_image::pixels, salmap, and env_dims::w.

void Saliency::waitUntilDoneWithInput ( ) const

Wait until process() is done using the input image.

This assumes that you are running process() in a different thread and here just want to wait until the initial processing that uses the input image is complete, so you can return that input image to the camera driver.

Definition at line 217 of file Saliency.C.

Friends And Related Function Documentation

void drawGist ( jevois::RawImage img,
unsigned char const *  gist,
size_t  gistsize,
unsigned int  xoff,
unsigned int  yoff,
unsigned int  width,
unsigned int  scale 
)
related

Draw a gist vector in a YUYV image as a rectangle of width width*scale and correct height.

Definition at line 771 of file Saliency.C.

void drawMap ( jevois::RawImage img,
env_image const *  fmap,
unsigned int  xoff,
unsigned int  yoff,
unsigned int  scale 
)
related

Draw a saliency map or feature map in a YUYV image.

Definition at line 709 of file Saliency.C.

void drawMap ( jevois::RawImage img,
env_image const *  fmap,
unsigned int  xoff,
unsigned int  yoff,
unsigned int  scale,
unsigned int  bitshift 
)
related

Draw a saliency map or feature map in a YUYV image, applying some right bitshift to values.

Definition at line 740 of file Saliency.C.

#define JEVOIS_DECLARE_PARAMETER (   ParamName,
  ParamType,
  ... 
)
related

Parameter.

#define JEVOIS_DECLARE_PARAMETER (   ParamName,
  ParamType,
  ... 
)
related

Parameter.

#define JEVOIS_DECLARE_PARAMETER (   ParamName,
  ParamType,
  ... 
)
related

Parameter.

#define JEVOIS_DECLARE_PARAMETER (   ParamName,
  ParamType,
  ... 
)
related

Parameter.

#define JEVOIS_DECLARE_PARAMETER (   ParamName,
  ParamType,
  ... 
)
related

Parameter.

#define JEVOIS_DECLARE_PARAMETER (   ParamName,
  ParamType,
  ... 
)
related

Parameter.

#define JEVOIS_DECLARE_PARAMETER (   ParamName,
  ParamType,
  ... 
)
related

Parameter.

#define JEVOIS_DECLARE_PARAMETER (   ParamName,
  ParamType,
  ... 
)
related

Parameter.

#define JEVOIS_DECLARE_PARAMETER (   ParamName,
  ParamType,
  ... 
)
related

Parameter.

#define JEVOIS_DECLARE_PARAMETER (   ParamName,
  ParamType,
  ... 
)
related

Parameter.

#define JEVOIS_DECLARE_PARAMETER (   ParamName,
  ParamType,
  ... 
)
related

Parameter.

Member Data Documentation

struct env_image Saliency::color

Definition at line 127 of file Saliency.H.

Referenced by process(), and Saliency().

struct env_image Saliency::flicker

Definition at line 129 of file Saliency.H.

Referenced by process(), and Saliency().

unsigned char* Saliency::gist

Gist vector has 1152 entries, 72 feature maps * 16 values/map.

The 16 values are in raster order. If a feature weight is zero, the corresponding gist values are all zero. The feature maps are in order:

red/green (6*16), blue/yellow (6*16), intens (6*16), ori (6*4*16), flicker (6*16), motion (6*4*16)

so the offsets are: red/green: offset 0 len 6*16 blue/yellow: offset 1*6*16 len 6*16 intens: offset 2*6*16 len 6*16 ori0: offset 3*6*16 len 6*16 ori1: offset 4*6*16 len 6*16 ori2: offset 5*6*16 len 6*16 ori3: offset 6*6*16 len 6*16 flicker: offset 7*6*16 len 6*16 motion0: offset 8*6*16 len 6*16 motion1: offset 9*6*16 len 6*16 motion2: offset 10*6*16 len 6*16 motion3: offset 11*6*16 len 6*16

Definition at line 151 of file Saliency.H.

Referenced by Saliency(), and ~Saliency().

size_t const Saliency::gist_size

Definition at line 152 of file Saliency.H.

Referenced by Saliency().

struct env_image Saliency::intens

Definition at line 126 of file Saliency.H.

Referenced by process(), and Saliency().

struct env_image Saliency::motion

Definition at line 130 of file Saliency.H.

Referenced by process(), and Saliency().

struct env_image Saliency::ori

Definition at line 128 of file Saliency.H.

Referenced by process(), and Saliency().

struct env_image Saliency::salmap

The saliency map.

Definition at line 118 of file Saliency.H.

Referenced by getSaliencyMax(), inhibitionOfReturn(), process(), and Saliency().


The documentation for this class was generated from the following files: