Adding new Python libraries from source

This tutorial explores how to add new functionality to JeVois in the form of libraries that are accessible from Python. We use the example of libdmtx, a library to decode DataMatrix tags.

Goals

We want to be able to decode DataMatrix tags in Python modules running on JeVois.

Obstacles

• As is usually the case with machine vision and A.I., the core decoding algorithm for this problem was written in C/C++. So we will need to compile it natively to run on the host computer for testing, and then we will need to also cross-compile it to run on the ARM processor inside JeVois.
• In addition to a compiled C library that implements the core functionality, we need Python bindings so that we can make calls to the library from Python.
• In addition to getting the core library and its Python bindings, we will need to write a JeVois module in Python that imports that library and uses it in a meaningful way.

Plan of attack

We first need to identify the relevant source code. A bit of web searching in this case suggests:

• libdmtx for the core algorithm and library written in C, at https://github.com/dmtx/libdmtx
• pylibdmtx for the python bindings, at https://github.com/NaturalHistoryMuseum/pylibdmtx

Looking at the examples provided with pylibdmtx, it looks like it will support numpy images as used by OpenCV, which is great since this is the image format used by Python modules in JeVois. So we should be able to easyly write a JeVois Python module that receives video frames from the camera sensor as OpenCV numpy arrays and processes them using pylibdmtx (invoking the libdmtx core algorithm).

Getting the source code of the libraries

For this tutorial, we will add the core library, python bindings, and JeVois module to the jevoisextra repository. You are invited to do the same and then let us know once you have something that works great and send us a pull request so we can integrate your new libraries and modules for all to enjoy.

• Grab the latest jevoisexra if you don't already have it:

  cd
  git clone https://github.com/jevois/jevoisextra.git
  

• jevoisextra has a mechanism to fetch external source code dependencies, which is in Contrib:
  • reinstall.sh downloads all dependencies
  • check.sh checks whether dependencies should be downloaded, by comparing the number in file RELEASE (part of the repository and updated manually each time new dependencies are added) to that in file .installed (generated by reinstall.sh when it runs).

Here we add the following to reinstall.sh:

###################################################################################################
# Cleanup:
/bin/rm -rf libdmtx pylibdmtx
 
###################################################################################################
# Get the packages:
 
# Core library to decode DataMatrix tags
get_github dmtx libdmtx 0c8cb2f542e74ee49ea5b2290a9c60d69b74fc01
 
# Python bindings to libdmtx
get_github NaturalHistoryMuseum pylibdmtx 6b1bb59fc7b0c55c56bc09eb6ded173144149555

The syntax for get_github is creator name, repo name, and commit version. It is desirable to use a specific commit version so that users do not get bad surprises if they run this script at a later time when the dependencies may have changed significantly and the head revision may not be compatible with our code anymore.

Then we increase the version number in RELEASE and just run:

./reinstall.sh

Compiling and cross-compiling the core C/C++ library

To allow for concurrent native compilation and cross-compilation, we will use the CMake build tool, and the CMake macros and scripts provided by the JeVois framework. We will compile libdmtx in a similar way as libjevoisbase is compiled in JeVois.

One hurdle is that libdmtx does not use CMake for building, rather it uses autoconf and a configure script plus a Makefile. Since this project is not very complex, we will just here bypass the configure script and write our own CMakeLists.txt instead. We just need to figure out, by inspecting the configure script and the makefile:

• where include files are located
• which C source files should be compiled into the library
• any flags that are set during the configure process and that we should also set for the library to achieve full functionality.

In this particular case, looking at configure.ac of libdmtx, we note:

• we are going to need to link against libm for some math functions like sin(), cos(), etc.
• we need gettimeofday to work.
• a variable ARCH is defined; but using grep to find any use of it in any of teh source code returns nothing, so we will just ignore it for now.

Now, looking at Makefile.am, we see:

• only one header: dmtx.h
• source files: dmtx.c, which, upon inspection, will include all the .c files listed under EXTRA_libdmtx_la_SOURCES
• some bening CPPFLAGS are provided, let's see if we can get away with our defaults instead

Using the CMakeLists.txt of repository jevoisbase as an example (since it also builds a library, libjevoisbase.so), we create the following new CMakeLists.txt

Since Contrib/libdmtx/ is an externally-fetched repository, let use another location for our CMakeLists.txt. To avoid confusion and possible interference with other versions of libdmtx that might be installed on the host machine (e.g., from Ubunty packages), we will call our library libjvdmtx.so as opposed to plain libdmtx.so:

cd ~/jevoisextra
mkdir libjvdmtx
cd libjvdmtx

And in there we create CMakeLists.txt as follows:

# CMake build rules for libdmtx library
 
# You may provide the installed JeVois config root and version as:
#   cmake -DJEVOIS_CONFIG=/jevois/config ..
 
cmake_minimum_required(VERSION 3.1)
 
# Set vendor name, our modules will be placed in a directory by that name under /jevois/modules:
set(JEVOIS_VENDOR "JeVois")
set(JEVOIS_CONFIG "/jevois/config" CACHE STRING "Path to JeVois config to use")
 
# Include our helper functions, config, etc from the JeVois install:
set(CMAKE_MODULE_PATH ${JEVOIS_CONFIG})
include(jevois_config)
include(JeVois)
 
# Set project name, detects compiler (which has been set by our helper module). Then set some complation flags:
project(jvdmtx C)
jevois_project_set_flags()
 
# Setup our library using just dmtx.c, which includes all other source files, and call it libjvdmtx with
# a version number:
 
# Note: jevois_setup_library2() expects that only one source file needs tobe compiled. You can add moe later
# using target_sources() as is done in the CMakeLists.txt of jevoisbase. If you want to compile all the C/C++
# files in a directory into the library, use jevois_setup_library() as is done in jevoisbase.
jevois_setup_library2("${CMAKE_CURRENT_SOURCE_DIR}/../Contrib/libdmtx/dmtx.c" jvdmtx 1.0.0)
 
# Add includes to our install / distribution package:
#add_subdirectory(include)
 
# libdmtx needs libm:
target_link_libraries(jvdmtx m)
 
# Let the .c files of libdmtx find dmtx.h:
include_directories(../Contrib/libdmtx)
 
# We could optionally add some definitions here:
#add_definitions(-DDARKNET_NNPACK)

We also copy rebuild-host.sh, rebuild-platform.sh, COPYING, INSTALL and README from the jevoisbase repository and edit them to adapt them to this project.

Lets' try to compile for host:

itti@iLab0:~/jevoisextra/libjvdmtx$ ./rebuild-host.sh
-- JeVois version 1.11.0
-- JEVOIS_PLATFORM: OFF
-- JEVOIS_VENDOR: JeVois
-- JeVois microSD card mount point: /media/itti/JEVOIS
-- JeVois serial-over-USB device: /dev/ttyACM0
-- Install prefix for executable programs: /usr
-- Host path to jevois modules root: /jevois
-- The C compiler identification is GNU 7.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- JeVois SDK root: /lab/itti/jevois/software/jevois-sdk
-- Host path to jevois lib and data install root: /jevois
-- Configuring done
-- Generating done
-- Build files have been written to: /lab/itti/jevois/software/jevoisextra/libjvdmtx/hbuild
Scanning dependencies of target jvdmtx
[ 50%] Building C object CMakeFiles/jvdmtx.dir/lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtx.c.o
In file included from /lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtx.c:71:0:
/lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtxdecode.c: In function ‘dmtxDecodeCreateDiagnostic’:
/lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtxdecode.c:492:84: warning: parameter ‘style’ set but not used [-Wunused-but-set-parameter]
 dmtxDecodeCreateDiagnostic(DmtxDecode *dec, int *totalBytes, int *headerBytes, int style)
                                                                                    ^~~~~
In file included from /lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtx.c:78:0:
/lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtxreedsol.c: In function ‘RsDecode’:
/lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtxreedsol.c:151:48: warning: unused parameter ‘fix’ [-Wunused-parameter]
 RsDecode(unsigned char *code, int sizeIdx, int fix)
                                                ^~~
In file included from /lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtx.c:83:0:
/lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtxtime.c: In function ‘dmtxTimeNow’:
/lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtxtime.c:92:7: warning: suggest braces around empty body in an ‘if’ statement [-Wempty-body]
       ; /* XXX handle error better here */
       ^
[100%] Linking C shared library libjvdmtx.so
[100%] Built target jvdmtx
[100%] Built target jvdmtx
Install the project...
-- Install configuration: ""
-- Installing: /usr/lib/libjvdmtx.so.1.0.0
-- Installing: /usr/lib/libjvdmtx.so
-- Installing: /usr/share/doc/libjvdmtx-host/README
-- Installing: /usr/share/doc/libjvdmtx-host/INSTALL
-- Installing: /usr/share/doc/libjvdmtx-host/COPYING

It worked ok (just a couple warnings), and we end up with /usr/lib/libjvdmtx.so which we will later link against.

Now for platform:

itti@iLab0:~/jevoisextra/libjvdmtx$ ./rebuild-platform.sh --staging
-- JeVois version 1.11.0
-- JEVOIS_PLATFORM: ON
-- JEVOIS_VENDOR: JeVois
-- JeVois microSD card mount point: /media/itti/JEVOIS
-- JeVois serial-over-USB device: /dev/ttyACM0
-- JEVOIS_MODULES_TO_STAGING: ON
-- JEVOIS_MODULES_TO_MICROSD: OFF
-- JEVOIS_MODULES_TO_LIVE: OFF
-- Install prefix for executable programs: /var/lib/jevois-build/usr
-- Host path to jevois modules root: /var/lib/jevois-microsd
-- The C compiler identification is GNU 7.2.0
-- Check for working C compiler: /lab/itti/jevois/software/jevois-sdk/out/sun8iw5p1/linux/common/buildroot/host/usr/bin/arm-buildroot-linux-gnueabihf-gcc
-- Check for working C compiler: /lab/itti/jevois/software/jevois-sdk/out/sun8iw5p1/linux/common/buildroot/host/usr/bin/arm-buildroot-linux-gnueabihf-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- JeVois SDK root: /lab/itti/jevois/software/jevois-sdk
-- Host path to jevois lib and data install root: /var/lib/jevois-microsd
-- Configuring done
-- Generating done
-- Build files have been written to: /lab/itti/jevois/software/jevoisextra/libjvdmtx/pbuild
Scanning dependencies of target jvdmtx
[ 50%] Building C object CMakeFiles/jvdmtx.dir/lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtx.c.o
In file included from /lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtx.c:71:0:
/lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtxdecode.c: In function ‘dmtxDecodeCreateDiagnostic’:
/lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtxdecode.c:492:84: warning: parameter ‘style’ set but not used [-Wunused-but-set-parameter]
 dmtxDecodeCreateDiagnostic(DmtxDecode *dec, int *totalBytes, int *headerBytes, int style)
                                                                                    ^~~~~
In file included from /lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtx.c:78:0:
/lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtxreedsol.c: In function ‘RsDecode’:
/lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtxreedsol.c:151:48: warning: unused parameter ‘fix’ [-Wunused-parameter]
 RsDecode(unsigned char *code, int sizeIdx, int fix)
                                                ^~~
In file included from /lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtx.c:83:0:
/lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtxtime.c: In function ‘dmtxTimeNow’:
/lab/itti/jevois/software/jevoisextra/Contrib/libdmtx/dmtxtime.c:92:7: warning: suggest braces around empty body in an ‘if’ statement [-Wempty-body]
       ; /* XXX handle error better here */
       ^
[100%] Linking C shared library libjvdmtx.so
[100%] Built target jvdmtx
[100%] Built target jvdmtx
Install the project...
-- Install configuration: ""
-- Installing: /var/lib/jevois-microsd/lib/JeVois/libjvdmtx.so
-- Set runtime path of "/var/lib/jevois-microsd/lib/JeVois/libjvdmtx.so" to ""
-- Installing: /var/lib/jevois-build/usr/share/doc/libjvdmtx-platform/README
-- Installing: /var/lib/jevois-build/usr/share/doc/libjvdmtx-platform/INSTALL
-- Installing: /var/lib/jevois-build/usr/share/doc/libjvdmtx-platform/COPYING

Same warnings, and we end up with /var/lib/jevois-microsd/lib/JeVois/libjvdmtx.so which is cross-compiled for ARM.

Note

Ideally, we should also install dmtx.h somewhere, like in /usr/include, so that it can easily be found by code that wants to use the library. We have not fully figured that part out yet, so for now users of the library will have to