|
JeVois
1.22
JeVois Smart Embedded Machine Vision Toolkit
|
|
|
JeVois
1.22
JeVois Smart Embedded Machine Vision Toolkit
|
|
New users, be sure to check out JeVois Start.
Unlike a traditional camera that relies on a human viewer to interpret the contents of what it sees, JeVois runs powerful machine vision and deep learning algorithms right inside the camera. This allows you to directly connect JeVois-Pro to a custom robot, home automation system, drone, surveillance system, etc. You let JeVois-Pro handle the machine vision while you focus on developing the high-level functionality of your system.
JeVois is mainly a development and education tool: it provides a Linux-based programming environment (C++ or Python) that facilitates writing your own machine vision pipelines for real-time operation on the processor inside the camera.
The JeVois toolkit is primarily designed to run on embedded smart camera hardware, although it can also run on recent Linux-based computers, including Raspberry Pi. The toolkit facilitates image capture from a camera sensor, processing of the captured images through some machine vision algorithm, and sending the processed results to a host computer over a USB link (JeVois-A33) or displaying results using OpenGL (JeVois-Pro).
Two hardware models are available:
JeVois-A33 is a small USB-powered smart webcam. To the host computer, a JeVois-A33 smart camera appears like a regular USB webcam, and is fully plug-and-play with no driver installation needed on reasonably recent versions of Windows, MacOS, and Linux: A JeVois smart camera is compliant with the USB Video Class (UVC) revision 1.1. Like with regular USB webcams, users can select image resolution, framerate, and adjust camera settings such as brightness, exposure, color balance, etc. But a JeVois smart camera outputs images that are the results of processing camera frames through some vision algorithm, as opposed to simply outputing the raw camera frames like a regular USB webcam.
The JeVois smart camera is a compact (39mm x 32mm x 19mm, or 1.5" x 1.25" x 0.75"), USB-powered, self-contained and powerful quad-core ARM computer running the Linux operating system.
JeVois-Pro is a full Linux-based computer in a small form factor. In addition to a 6-core CPU, it integrates a variety of hardware accelerators for deep neural networks. Because it is a full computer, the primary output for JeVois-Pro is an HDMI display. There is no need for a host computer, as one can directly connect keyboard, mouse, and display to JeVois-Pro.
JeVois-A33 and JeVois-Pro run out of the box with all hardware and software pre-installed and pre-configured!
You may be able to piece together an equivalent system from parts, but it would likely be larger, heavier, and ugly/unreliable, with some loose parts such as a camera sensor or an accelerometer (IMU) hanging from some wires. This would also require a lot of work, both on the hardware side, and especially on the software side. As we have learned over the years, it is quite difficult to make all the hardware and software components inside JeVois-A33 and JeVois-Pro work seemlessly together.
A recent example is MediaPipe, a great library for deep networks that can detect hands, facial landmarks, etc. It required older dependencies and an older version of Python than other software libraries we also run. We had to update their source code, to reconfigure it, and to recompile it from source to make it work, which was far from easy. Likewise for Ollama (generative AI framework), PyCoral (Python libraries for Google Coral EdgeTPU), and many more. Getting all of those to work on one machine is indeed a feat!
For ease of programming and configuration, all of the operating system, core JeVois software, and any necessary data files are stored on a single high-speed Micro-SD card that can easily be removed and plugged into a desktop or laptop computer. This also makes the platform "unbrickable" and very hacking-friendly.
In addition to video outputs, a JeVois smart camera includes one hardware serial port and one serial-over-USB port (JeVois-Pro also includes additional hardware ports). Simple text strings can be sent by the smart camera over these ports, for example to indicate the coordinates at which an object of interest was found in the camera video stream. These are intended for use by embedded controllers, including Arduino or any other that supports serial communications (UART with TTL levels at 3.3V or 5V). Users or micro-controllers can also send simple text commands to the smart camera over the serial ports.
The JeVois software framework combines custom Linux kernel drivers for camera sensor and for USB output, written in C, and a custom high-level vision processing framework, written in C++-26. This framework is designed for easy integration with other open-source libraries, including tiny-dnn, OpenCV, boost, zBar, Eigen, turbojpeg, etc. The JeVois operating system infrastructure is built using the buildroot framework, for which hundreds of libraries and modules are already available, and adding new libraries or modules is relatively easy. The core JeVois software is managed by cmake, which makes it easy to create new vision processing modules, and which allows simultaneous compilation of the same source code for a host computer (e.g., Intel-based Linux desktop used for development, debugging and testing) and for the platform hardware (ARM processor).
For tutorials on how to use and program JeVois, see the JeVois Tutorials.
1.9.8
