JeVoisBase  1.18
JeVois Smart Embedded Machine Vision Toolkit Base Modules
Share this page:
Surprise.H
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #pragma once
19 
21 
22 namespace surprise
23 {
24  static jevois::ParameterCategory const ParamCateg("Surprise Options");
25 
26  //! Parameter \relates Surprise
27  JEVOIS_DECLARE_PARAMETER(updatefac, float, "Surprise update factor on every video frame", 0.95F,
28  jevois::Range<float>(0.001F, 0.999F), ParamCateg);
29 
30  //! Parameter \relates Surprise
31  JEVOIS_DECLARE_PARAMETER(channels, std::string, "Channels to use for surprise computation: any combination of "
32  "S (saliency), G (gist), C (color), I (intensity), O (orientation), F (flicker), and "
33  "M (motion). Duplicate letters will be ignored.",
34  "SCIOFMG", boost::regex("^[SCIOFMG]+$"), ParamCateg);
35 }
36 
37 
38 //! Compute Itti & Baldi surprise over video frames
39 /*! This component detects surprising events in video frames using Itti & Baldi's Bayesian theory of surprise.
40 
41  They defined surprise in a formal, quantitative manner (for the first time!), as follows: An observation is
42  surprising if it significantly affects the internal (subjective) beliefs of an observer. For example, if I believe
43  that there is a 10% chance of rain today (my prior belief), and then I look outside and I see only a few small
44  scattered clouds, then I may still believe in that same 10% chance of rain (posterior belief after the
45  observation). My observation was not surprising, and Itti & Baldi say that this is because it did not affect my
46  beliefs. Formally, when my posterior beliefs after an observation are very similar to what my prior beliefs were
47  before the observation, the observation carries no surprise. In contrast, if I see a sky covered with menacing dark
48  clouds all over, I may revise my belief to a 80% chance of rain today. Because my posterior beliefs are now much
49  different than my prior beliefs, the observation of clouds is said to carry a high surprise. Itti & Baldi further
50  specify how to compute surprise by using Bayes' theorem to compute posterior beliefs in a pricipled way, and by
51  using the Kullback-Leibler divergence to measure the difference between posterior and prior distributions of
52  beliefs. This gives rise to a new quantitative measure of surprise, with a new unit, the 'wow' (one wow of surprise
53  is experienced when your belief in something doubles).
54 
55  For more information, check out [L. Itti, P. F. Baldi, Bayesian Surprise Attracts Human Attention, Vision Research,
56  Vol. 49, No. 10, pp. 1295-1306, May 2009.](http://ilab.usc.edu/publications/doc/Itti_Baldi09vr.pdf)
57 
58  In this component, we compute feature maps and a saliency map. These will provide some degree of invariance and
59  robustness to noise, which will yield more stable overall results than if we were to compute surprise directly over
60  RGB pixel values (see next point);
61 
62  We then compute surprise in each pixel of each feature map. This is similar to what Itti & Baldi did but simplified
63  to run in real time on the JeVois smart camera. Each pixel in each feature map will over time gather beliefs about
64  what it usually 'sees' at that location in the video. When things change significantly and in a surprising way, that
65  pixel will emit a local surprise signal. Because surprise is more complex than just computing an instantaneous
66  difference, or measuring whether the current observation simply is an outlier to a learned distribution, it will be
67  able to handle periodic motions (foliage in the wind, ripples on a body of water), periodic flickers (a constantly
68  blinking light in the field of view), and noise.
69 
70  This approach is related to [R. C. Voorhies, L. Elazary, L. Itti, Neuromorphic Bayesian Surprise for Far-Range Event
71  Detection, In: Proc. 9th IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS),
72  Beijing, China, Sep 2012.](http://ilab.usc.edu/publications/doc/Voorhies_etal12avss.pdf).
73 
74  \ingroup components */
75 class Surprise : public jevois::Component,
76  public jevois::Parameter<surprise::updatefac, surprise::channels>
77 {
78  public:
79  //! Constructor
80  Surprise(std::string const & instance);
81 
82  //! Virtual destructor for safe inheritance
83  ~Surprise();
84 
85  //! Compute surprise from a YUYV video frame and return the surprise value in wows
86  double process(jevois::RawImage const & input);
87 
88  protected:
89  std::shared_ptr<Saliency> itsSaliency;
90  std::vector<float> itsAlpha, itsBeta;
91 };
92 
93 
Surprise::itsBeta
std::vector< float > itsBeta
Definition: Surprise.H:90
jevois::Range
Surprise::process
double process(jevois::RawImage const &input)
Compute surprise from a YUYV video frame and return the surprise value in wows.
Definition: Surprise.C:559
JEVOIS_DECLARE_PARAMETER
JEVOIS_DECLARE_PARAMETER(thresh1, double, "First threshold for hysteresis", 50.0, ParamCateg)
jevois::Component
jevois::RawImage
Surprise::~Surprise
~Surprise()
Virtual destructor for safe inheritance.
Definition: Surprise.C:28
jevois::ParameterCategory
Surprise
Compute Itti & Baldi surprise over video frames.
Definition: Surprise.H:75
F
float F
Surprise::Surprise
Surprise(std::string const &instance)
Constructor.
Definition: Surprise.C:21
surprise
Definition: Surprise.H:22
Surprise::itsAlpha
std::vector< float > itsAlpha
Definition: Surprise.H:90
Surprise::itsSaliency
std::shared_ptr< Saliency > itsSaliency
Definition: Surprise.H:89
Saliency.H