Surprise Recorder
Surprise-based recording of events.
By Laurent Ittiitti@usc.eduhttp://jevois.orgGPL v3
 Language: C++Supports mappings with USB output: NoSupports mappings with NO USB output: Yes 
 Video Mapping:   NONE 0 0 0 YUYV 640 480 15.0 JeVois SurpriseRecorder
This module has no screenshots and no videos

Module Documentation

This module detects surprising events in the live video feed from the camera, and records short video clips of each detected event.

Surprising is here defined according to Itti and Baldi's mathematical theory of surprise (see, e.g., http://ilab.usc.edu/surprise/) which is applied to monitoring live video streams. When a surprising event is detected, a short video clip of that event is saved to the microSD card inside JeVois, for later review.

It was created in this JeVois tutorial: http://jevois.org/tutorials/ProgrammerSurprise.html

Using this module

This module does not send any video output to USB. Rather, it just saves surprising events to microSD for later review. Hence, you may want to try the following:

  • mount the JeVois camera where you want it to detect surprising events
  • run it connected to a laptop computer, using any mode which does have some video output over USB (e.g., 640x500 YUYV). Adjust the camera orientation to best fit your needs.
  • edit JEVOIS:/config/initscript.cfg to contain:
      setmapping2 YUYV 640 480 15.0 JeVois SurpriseRecorder
      setpar thresh 1e7
      setpar channels S
      streamon
    and see the above tutorial for more details. Next time you power JeVois, it will immediately start detecting and recording surprising events in its view.

Example

Here is one hour of video surveillance footage. It is very boring overall. Except that a few brief surprising things occur (a few seconds each). Can you find them?

Here is what the SurpriseRecorder module found (4 true events plus 2 false alarms):

With only 6 surprising events, and assuming +/- 10 seconds of context frames around each event, we have achieved a compression of the surveillance footage from 60 minutes to 2 minutes (a factor 30x).

ParameterTypeDescriptionDefaultValid Values
(SurpriseRecorder) filenamestd::stringName of the video file to write. If path is not absolute, /jevois/data/surpriserecorder/ will be prepended to it. Name should contain a printf-like directive for one int argument, which will start at 0 and be incremented on each streamoff command.video%06d.avi-
(SurpriseRecorder) fourccstd::stringFourCC of the codec to use. The OpenCV VideoWriter doc is unclear as to which codecs are supported. Presumably, the ffmpeg library is used inside OpenCV. Hence any video encoder supported by ffmpeg should work. Tested codecs include: MJPG, MP4V, AVC1. Make sure you also pick the right filename extension (e.g., .avi for MJPG, .mp4 for MP4V, etc)MJPGboost::regex(^\\\\w{4}$)
(SurpriseRecorder) fpsdoubleVideo frames/sec as stored in the file and to be used both for recording and playback. Beware that the video writer will drop frames if you are capturing faster than the frame rate specified here. For example, if capturing at 120fps, be sure to set this parameter to 120, otherwise by default the saved video will be at 30fps even though capture was running at 120fps.15.0-
(SurpriseRecorder) threshdoubleSurprise threshold. Lower values will record more events.1.0e7-
(SurpriseRecorder) ctxframesunsigned intNumber of context video frames recorded before and after each surprising event.150-
(Saliency) cweightbyteColor channel weight255-
(Saliency) iweightbyteIntensity channel weight255-
(Saliency) oweightbyteOrientation channel weight255-
(Saliency) fweightbyteFlicker channel weight255-
(Saliency) mweightbyteMotion channel weight255-
(Saliency) centerminsize_tLowest (finest) of the 3 center scales2-
(Saliency) deltaminsize_tLowest (finest) of the 2 center-surround delta scales3-
(Saliency) smscalesize_tScale of the saliency map4-
(Saliency) mthreshbyteMotion threshold0-
(Saliency) fthreshbyteFlicker threshold0-
(Saliency) msflickboolUse multiscale flicker computationfalse-
(Surprise) updatefacfloatSurprise update factor on every video frame0.95Fjevois::Range<float>(0.001F, 0.999F)
(Surprise) channelsstd::stringChannels to use for surprise computation: any combination of S (saliency), G (gist), C (color), I (intensity), O (orientation), F (flicker), and M (motion). Duplicate letters will be ignored.SCIOFMGboost::regex(^[SCIOFMG]+$)
Detailed docs:SurpriseRecorder
Copyright:Copyright (C) 2016 by Laurent Itti, iLab and the University of Southern California
License:GPL v3
Distribution:Unrestricted
Restrictions:None
Support URL:http://jevois.org/doc
Other URL:http://iLab.usc.edu
Address:University of Southern California, HNB-07A, 3641 Watt Way, Los Angeles, CA 90089-2520, USA