JeVoisBase
1.20
JeVois Smart Embedded Machine Vision Toolkit Base Modules
Share this page:
Tweet
Surprise.H
Go to the documentation of this file.
1
// ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2
//
3
// JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4
// California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5
//
6
// This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7
// redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8
// Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9
// without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10
// License for more details. You should have received a copy of the GNU General Public License along with this program;
11
// if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12
//
13
// Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14
// Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15
// ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16
/*! \file */
17
18
#pragma once
19
20
#include <
jevoisbase/Components/Saliency/Saliency.H
>
21
22
namespace
surprise
23
{
24
static
jevois::ParameterCategory
const
ParamCateg(
"Surprise Options"
);
25
26
//! Parameter \relates Surprise
27
JEVOIS_DECLARE_PARAMETER
(updatefac,
float
,
"Surprise update factor on every video frame"
, 0.95
F
,
28
jevois::Range<float>
(0.001
F
, 0.999
F
), ParamCateg);
29
30
//! Parameter \relates Surprise
31
JEVOIS_DECLARE_PARAMETER
(channels, std::string,
"Channels to use for surprise computation: any combination of "
32
"S (saliency), G (gist), C (color), I (intensity), O (orientation), F (flicker), and "
33
"M (motion). Duplicate letters will be ignored."
,
34
"SCIOFMG"
, boost::regex(
"^[SCIOFMG]+$"
), ParamCateg);
35
}
36
37
38
//! Compute Itti & Baldi surprise over video frames
39
/*! This component detects surprising events in video frames using Itti & Baldi's Bayesian theory of surprise.
40
41
They defined surprise in a formal, quantitative manner (for the first time!), as follows: An observation is
42
surprising if it significantly affects the internal (subjective) beliefs of an observer. For example, if I believe
43
that there is a 10% chance of rain today (my prior belief), and then I look outside and I see only a few small
44
scattered clouds, then I may still believe in that same 10% chance of rain (posterior belief after the
45
observation). My observation was not surprising, and Itti & Baldi say that this is because it did not affect my
46
beliefs. Formally, when my posterior beliefs after an observation are very similar to what my prior beliefs were
47
before the observation, the observation carries no surprise. In contrast, if I see a sky covered with menacing dark
48
clouds all over, I may revise my belief to a 80% chance of rain today. Because my posterior beliefs are now much
49
different than my prior beliefs, the observation of clouds is said to carry a high surprise. Itti & Baldi further
50
specify how to compute surprise by using Bayes' theorem to compute posterior beliefs in a pricipled way, and by
51
using the Kullback-Leibler divergence to measure the difference between posterior and prior distributions of
52
beliefs. This gives rise to a new quantitative measure of surprise, with a new unit, the 'wow' (one wow of surprise
53
is experienced when your belief in something doubles).
54
55
For more information, check out [L. Itti, P. F. Baldi, Bayesian Surprise Attracts Human Attention, Vision Research,
56
Vol. 49, No. 10, pp. 1295-1306, May 2009.](http://ilab.usc.edu/publications/doc/Itti_Baldi09vr.pdf)
57
58
In this component, we compute feature maps and a saliency map. These will provide some degree of invariance and
59
robustness to noise, which will yield more stable overall results than if we were to compute surprise directly over
60
RGB pixel values (see next point);
61
62
We then compute surprise in each pixel of each feature map. This is similar to what Itti & Baldi did but simplified
63
to run in real time on the JeVois smart camera. Each pixel in each feature map will over time gather beliefs about
64
what it usually 'sees' at that location in the video. When things change significantly and in a surprising way, that
65
pixel will emit a local surprise signal. Because surprise is more complex than just computing an instantaneous
66
difference, or measuring whether the current observation simply is an outlier to a learned distribution, it will be
67
able to handle periodic motions (foliage in the wind, ripples on a body of water), periodic flickers (a constantly
68
blinking light in the field of view), and noise.
69
70
This approach is related to [R. C. Voorhies, L. Elazary, L. Itti, Neuromorphic Bayesian Surprise for Far-Range Event
71
Detection, In: Proc. 9th IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS),
72
Beijing, China, Sep 2012.](http://ilab.usc.edu/publications/doc/Voorhies_etal12avss.pdf).
73
74
\ingroup components */
75
class
Surprise
:
public
jevois::Component
,
76
public
jevois::Parameter<surprise::updatefac, surprise::channels>
77
{
78
public
:
79
//! Constructor
80
Surprise
(std::string
const
& instance);
81
82
//! Virtual destructor for safe inheritance
83
~Surprise
();
84
85
//! Compute surprise from a YUYV video frame and return the surprise value in wows
86
double
process
(
jevois::RawImage
const
& input);
87
88
protected
:
89
std::shared_ptr<Saliency>
itsSaliency
;
90
std::vector<float>
itsAlpha
,
itsBeta
;
91
};
92
93
Surprise::itsBeta
std::vector< float > itsBeta
Definition:
Surprise.H:90
jevois::Range
Surprise::process
double process(jevois::RawImage const &input)
Compute surprise from a YUYV video frame and return the surprise value in wows.
Definition:
Surprise.C:559
JEVOIS_DECLARE_PARAMETER
JEVOIS_DECLARE_PARAMETER(thresh1, double, "First threshold for hysteresis", 50.0, ParamCateg)
jevois::Component
jevois::RawImage
Surprise::~Surprise
~Surprise()
Virtual destructor for safe inheritance.
Definition:
Surprise.C:28
jevois::ParameterCategory
Surprise
Compute Itti & Baldi surprise over video frames.
Definition:
Surprise.H:75
F
float F
Surprise::Surprise
Surprise(std::string const &instance)
Constructor.
Definition:
Surprise.C:21
surprise
Definition:
Surprise.H:22
Surprise::itsAlpha
std::vector< float > itsAlpha
Definition:
Surprise.H:90
Surprise::itsSaliency
std::shared_ptr< Saliency > itsSaliency
Definition:
Surprise.H:89
Saliency.H
include
jevoisbase
Components
Saliency
Surprise.H
Please help us improve this page: Edit it on
GitHub
or
email us your edits and suggestions.
Generated by
1.8.17