Another member here noted that the red LED is quite weak compared to the green, maybe yours is extra weak? did you try looking directly into the LED hole in the case?
To get the different processing modes in AMCap, you need to select different resolutions and video modes. The ones that have markings on the video are mostly in YUYV pixel format (AMCap calls it YUY2), various resolutions. Please see
http://jevois.org/doc/UserStartWindows.html for how to change resolution in AMCap
http://jevois.org/start/start.html under "demos" for info about which algorithm runs for different resolutions