Demo Saliency + Gist + Face Detection + Object Recognition
Simple demo that combines saliency, gist, face detection, and object recognition.
By Laurent Ittiitti@usc.eduhttp://jevois.orgGPL v3
Video Mapping: YUYV 640 312 50.0 YUYV 320 240 50.0 JeVois DemoSalGistFaceObj

Run the visual saliency algorithm to find the most interesting location in the field of view. Then extract a square image region around that point. On alternating frames, either

  • attempt to detect a face in the attended region, and, if positively detected, show the face in the bottom-right corner of the display. The last detected face will remain shown in the bottom-right corner of the display until a new face is detected.
  • or attempt to recognize an object in the attended region, using a deep neural network. The default network is a handwritten digot recognition network that replicated the original LeNet by Yann LeCun and is one of the very first convolutional neural networks. The network has been trained on the standard MNIST database of handwritten digits, and achives over 99% correct recognition on the MNIST test dataset. When a digit is positively identified, a picture of it appears near the last detected face towards the bottom-right corner of the display, and a text string with the digit that has been identified appears to the left of the picture of the digit.
ParameterTypeDescriptionDefaultValid Values
(Saliency) cweightbyteColor channel weight255-
(Saliency) iweightbyteIntensity channel weight255-
(Saliency) oweightbyteOrientation channel weight255-
(Saliency) fweightbyteFlicker channel weight255-
(Saliency) mweightbyteMotion channel weight255-
(Saliency) centerminsize_tLowest (finest) of the 3 center scales2-
(Saliency) deltaminsize_tLowest (finest) of the 2 center-surround delta scales3-
(Saliency) smscalesize_tScale of the saliency map4-
(Saliency) mthreshbyteMotion threshold0-
(Saliency) fthreshbyteFlicker threshold0-
(Saliency) msflickboolUse multiscale flicker computationfalse-
(FaceDetector) face_cascadestd::stringFile name of the face cascadefacedetector/haarcascade_frontalface_alt.xml-
(FaceDetector) eye_cascadestd::stringFile name of the eye cascade, or empty to not detect eyesfacedetector/haarcascade_eye_tree_eyeglasses.xml-
(Kalman2D) usevelboolUse velocity tracking, in addition to positionfalse-
(Kalman2D) procnoisefloatProcess noise standard deviation0.003F-
(Kalman2D) measnoisefloatMeasurement noise standard deviation0.05F-
(Kalman2D) postnoisefloatA posteriori error estimate standard deviation0.3F-
Detailed docs:DemoSalGistFaceObj, Saliency, FaceDetector, ObjectRecognitionMNIST, Kalman2D
Copyright:Copyright (C) 2016 by Laurent Itti, iLab and the University of Southern California
License:GPL v3
Support URL:
Other URL:
Address:University of Southern California, HNB-07A, 3641 Watt Way, Los Angeles, CA 90089-2520, USA