JeVois  1.23
JeVois Smart Embedded Machine Vision Toolkit
Share this page:
Loading...
Searching...
No Matches
JeVois-Pro Development Status

Updated: Setp 9, 2021

Legend

  • Implemented, tested and working:
  • Not yet implemented, or not working:

Current development status

Feature Status Notes
Hardware Design - -
All 19 internal power supplies operational from single 12-24 VDC input OK -
4x A73 big cores running at full speed with no errors under full load OK -
2x A53 little cores running at full speed with no errors under full load OK -
4GB LPDDR4 working with no error at full DDR4-3200 speed OK -
ARM NEON accelerated multimedia processor instructions tested and working OK -
A311D MALI GPU OpenGL-ES 3.2 working using ARM demos (framebuffer mode) OK -
Micro-SD / TF card working at SDR104 (104 MBytes/s) speed OK -
HDMI output, video + sound OK -
2x USB 2.0 type-A ports at top of camera provide power and are working OK -
Mini-USB 2.0 OTG port working, both in host and in device modes OK -
4-pin micro-serial port working OK -
6-pin AUX power port working (outputs 5V, 3.3V, 1.8V for accessories) OK -
8-pin GPIO port working (configurable as SPI+I2C, or general GPIO) OK (12)
Thermally stable operation under full load OK -
Camera sensor connector working, camera functionality OK -
Camera sensor connector working, dual-camera functionality ? (10)
Camera sensor connector working, SPI inertial measurement unit (IMU) functionality OK -
M.2 E-Key connector working with PCIe M.2 cards (Coral TPU, Wifi cards) OK -
M.2 E-Key connector working with USB/SDIO M.2 cards (Wifi/bluetooth cards) OK -
Wired ethernet using add-on USB-to-RJ45 converter OK -
Variable-speed cooling fan control OK -
Base System - -
Bootloader, Linux Kernel 4.9.x, partitions, Ubuntu 20.04 OK -
Custom kernel device tree OK -
Boot to X-windows and use as a regular computer when connecting keyboard/mouse OK -
Play Youtube videos with sound over HDMI OK -
Boot to console OK -
Boot to JeVois software OK -
JeVois-Pro video + serial + storage over mini-USB OTG, kernel Gadget driver NO (13)
JeVois-Pro control via JeVois Inventor NO (14)
Software reset OK -
Camera Sensor - -
Sony IMX-290 2MP camera sensor detection and basic 1080p @ 30fps operation OK -
Sony IMX-290 use 4x MIPI-CSI lanes for lowest latency OK -
Sony IMX-290 hardware-accelerated conversion from Bayer to YUYV, RGB, RGBA, GREY OK -
Sony IMX-290 support for basic controls (brightness, exposure, gain, etc) OK -
Sony IMX-290 frame cropping support, any size from native 1080p OK (11)
Sony IMX-290 ISP-based hardware-accelerated frame scaling support, any size OK (11)
Sony IMX-290 ISP-based dual-stream capture, e.g., YUYV 1920x1080 + RGB 512x288 OK -
Sony IMX-290 native 720p support NO (1)
Sony IMX-290 frame rate control NO (2)
IMU Sensor - -
TDK ICM-29048 inertial measurement uint (IMU) detection over 7MHz SPI bus OK -
TDK ICM-29048 IMU basic operation, accel, gyro, compass, temperature over 7MHz SPI OK -
TDK ICM-29048 IMU upload digital motion processor (DMP) code OK -
TDK ICM-29048 IMU run digital motion processor (DMP) code NO (3)
Machine Vision Base Software - -
OpenCV 4.5.3 + OpenVino 2021.4 + all OpenCV-Contrib + Python3 bindings OK -
OpenGL-ES 3.2 framebuffer OK -
OpenCL 1.2 OK -
Python 3.8 + numpy + scipy + any aarch64 python packages OK -
TensorFlow Lite 2.5 OK -
Mediapipe 0.8 python OK -
Vulkan, hardware H.265 encode/decode, etc ? (4)
JeVois Base Software - -
JeVois use OpenGL for 1080p display @ 30fps over HDMI OK -
JeVois OpenGL support for zero-copy video display using DMABUF, EGLImageKHR OK -
JeVois OpenGL DMABUF, EGLImageKHR support for RGB, RGBA, YUYV, GREY direct display OK -
JeVois OpenGL support for DMABUF display of YUYV, RGB, RGBA, GREY from camera OK -
JeVois OpenGL support for display of processed YUYV, RGB, RGBA, GREY using shaders OK (7)
JeVois capture fullsize+scaled streams from camera, display fullsize, process scaled OK -
JeVois new on-screen graphical user interface using Dear ImGui 1.83 OK -
JeVois new ImGui backend for MALI OpenGL-ES GPU in framebuffer mode OK -
JeVois new drivers to capture console keyboard/mouse events and pass to ImGui OK -
JeVois run OpenCV machine vision algorithms written in C++ OK -
JeVois run other machine vision algorithms written in C++ OK -
JeVois run OpenCV machine vision algorithms written in Python OK -
JeVois run other machine vision algorithms written in Python OK -
JeVois support for Python modules to draw OpenGL overlays (box, text, etc) OK -
JeVois-Pro run machine vision algorithms developed for JeVois-A33, C++ and Python OK -
JeVois-Pro enhance some JeVois-A33 algorithms to use DMABUF zero-copy + OpenGL OK -
JeVois new dual threadpool with user-selectable big/little affinity OK -
JeVois new async_big() and async_little() thread launching functions OK -
JeVois New DNN Software - -
New JeVois deep neural network (DNN) framework OK -
JeVois DNN framework support for classification, detection, semantic segmentation OK -
JeVois DNN framework hierarchical model zoo with 30+ pre-trained models OK -
JeVois DNN framework download and run custom models OK -
JeVois DNN run models on CPU using OpenCV DNN module OK -
JeVois DNN run models on A311D 5-TOPS integrated neural processing uint (NPU) OK -
JeVois custom kernel driver to enable Coral 4-TOPS accelerator over PCIe-2.1 (5 Gbps) OK -
JeVois DNN run models on Coral 4-TOPS accelerator, PCIe-2.1 (5 Gbps) M.2 card OK -
JeVois DNN run models on Coral 4-TOPS accelerator, USB-2.0 (480 Mbps) dongle OK -
JeVois DNN run models on Myriad-X 1-TOPS accelerator with OpenVino, PCIe-2.1 M.2 card OK -
JeVois DNN run models on Myriad-X 1-TOPS accelerator with OpenVino, USB-2.0 dongle OK -
JeVois DNN run models on CPU, A311D NPU, Coral TPU, Myriad-X VPU in parallel OK -
JeVois DNN run multiple models on one CPU, NPU, TPU, VPU hardware, time multiplexed OK -
Support for Coral M.2 board with 2xTPU chips using 2xPCIe lanes, made by Google NO (5)
Support for custom JeVois 2xTPU board with 1 TPU on PCIe and the other on USB OK -
Support for custom JeVois 2xTPU + eMMC board with PCIe hub + 2 TPUs on PCIe OK -

Notes:

  1. ARM ISP hardware reports frame collision when using IMX-290 datasheet's 720p settings. May be hardware incompatibility between ISP and sensor running at 720p, or bug in ARM's ISP kernel driver. Can still capture at 1080p and use hardware-accelerated rescaling to 720p.
  2. Need to add support for VIDIOC_S_PARM and VIDIOC_G_PARM in the ARM ISP kernel driver for V4L2. Currently frame rate is determined by the chosen IMX290 operation mode and sensor settings (default 1080p @ 30fps).
  3. May be a voltage translation issue on our prototype boards, or may be a software issue. A311D chip may be too weak to drive resistor-based voltage divider at 7MHz SPI speed. Hardware revision of our sensor board in progress using active voltage translator chip. For early prototypes, assume that DMP is not operational.
  4. These features have been shown to work on other development boards using the A311D chip, so they should work, possibly after instaling necessary drivers and other software.
  5. A331D only has 1x PCIe lane. Beware that you will only be able to access one TPU on JeVois if you use the Coral dual-TPU board developed and sold by the Coral team. We designed a dual-TPU board that connects both TPUs to the 1x PCIe bus of the A311D using a PCIe switch. The board works great.
  6. Some aliasing issues in the shaders, which seem to only exist on the A311D and not on our desktop GPUs, require that processed images be displayed with an integer scaling factor, likely due to lower precision of the A311D OpenGL-ES processor compared to desktop full OpenGL. For example, a 640x480 YUYV processing output will show aliasing artifacts if scaled to fit full screen height (from 640x480 to 1440x1080 pixels, scaling factor 2.25), but displays ok if scaled by a factor 2 (1280x960). This only affects "old-style" JeVois machine vision modules that created a new output image with the processing results drawn using pixel-based drawing functions into a copy of an input image. The preferred new style of processing on JeVois-Pro is to simply display the 1080p raw image captured from the camera using zero-copy DMABUF, then have machine vision modules draw ImGui/OpenGL overlays on top of it, letting OpenGL render the image plus overlays (with possible alpha blending). In such scenario, no aliasing issues exist.
  7. Required signals for dual camera sensors have been routed but not yet tested as we do not yet have a dual-sensor prototype board. Dual camera sensor boards can use 2 separate MIPI-CSI clocks, and up to 2 MIPI lanes for each sensor. Single camera sensor boards (e.g., default Sony IMX290 board) use one MIPI-CSI clock and up to 4 MIPI lanes.
  8. Supports cropping or scaling of camera frames to any size, but OpenGL, OpenCV, and others often require image width and height to be multiples of 32.
  9. See JeVois-Pro auxilliary connectors
  10. Since JeVois-Pro has an HDMI output, emulating a USB webcam like we did on JeVois-A33 is of lesser interest. During development of algorithms, you can just connect JeVois-Pro directly to display, keyboard, and mouse, and JeVois-Pro has a full graphical user interface that allows you to tweak parameters, run various machine vision modules, edit Python code, etc. For headless operation once an algorithm is finalized, use the 4-pin micro-serial port to output machine vision results as text strings (e.g., "face detected at X,Y", or "QRcode decoded value is Z", etc).
  11. JeVois-Inventor is a desktop app that allows one to control a JeVois-A33 smart camera, which has no display or keyboard/mouse. Since JeVois-Pro has display and keyboard/mouse, the functionality of JeVois-Inventor has been converted into a full graphical user interface that runs directly in JeVois-Pro, thereby eliminating the need for a host computer entirely.