Great idea. Faster-RCNN with inceptionv2 may be a bit too big to run on JeVois.
Have you tried YOLO? The tiny YOLO version can detect people (and 19 other things like bicycles, cars, etc) and runs ok on JeVois. You can either run the original code from the authors, or the OpenCV version:
for the OpenCV DNN version, a different version is loaded by default for face detection. So you need to edit params.cfg (for DetectionDNN) or the python code (for PyDetectionDNN) to swithc to tiny YOLO instead.