#include <jevois/Core/Module.H>

Virtual base class for a vision processing module.

Module is the base class to implement camera-to-USB frame-by-frame video processing. The Engine instantiates one class derived from Module, according to the current VideoMapping selected by the end user (e.g., current image resolution, format, and frame rate setected by a webcam viewing program on a host computer). The Module is loaded as shared object (.so) file according to the VideoMapping definitions in videomappings.cfg and the current VideoMapping selected by the user.

Module derives from Component and as such can contain:

any number of Parameter settings
any arbitrarily complex sub-hierarchy of Component objects to implement various functionality. Parameter settings from all the sub-components are available to the Module and to users connected over Serial ports of the Engine.

This allows one to implement complex vision processing pipelines efficiently and with substantial code re-use. For example, one may first want to implement an EdgeDetector or Saliency component, with Parameter settings for various thresholds, features, etc. One can then create any number of top-level objects that derive from Module and that may contain one or more EdgeDetector, Saliency, etc components in their hierarchy of sub-components, with the implementation in the module simply routing images from one Component to another to create a processing pipeline.

Classes that derive from Module should implement up to four functions:

process(InputFrame && inframe, OutputFrame && outframe) is called once per iteration of the Engine main loop when the current VideoMapping specifies both a particular Camera resolution and format, and a USB resolution and format. This function should process the received input frame and fill the pixel buffer of the output frame with results. Memory has already been allocated for both the input and output images before process() is called. Because the output image is actually allocated by the USB Gadget driver (and, ultimately, by the Linux kernel), its pixel memory location cannot be moved (hence, do not attempt to copy the output image or replace it by another image, etc; just write pixel data into the output image's pixel array). There is no restriction on video modes or frame rates, except as suported by the Camera hardware, and as limited by USB bandwidth. For most implementations, matching the input and output frame rate is easiest, and means that each invocation of process() would access and use both of the provided InputFrame and OutputFrame (one-input-to-one-output processing pipeline). But this is not mandatory. For example, a motion flow computation Module for use in a flying drone may have 320x240 YUYV 53.0fps inputs and 100x142 GREY 10.6fps output (since output frame rate is 5x lower than input in this example, the process() function would here get, fill, and send the OutputFrame only once every 5 times it is called; implementation of the process() function should keep track of that, e.g., through a member variable that gets incremented each time process() is called). In addition to filling the pixel data of the OutputFrame, process() may also send results over the serial ports (e.g., for use by an Arduino connected to the JeVois platform hardware) using sendSerial().
process(InputFrame && inframe) is called once per Camera frame when the current VideoMapping specifies a particular Camera resolution and format, and NONE as USB output format. This function should process the received input frame and would typicaly then send results to serial ports (e.g., for use by an Arduino connected to the JeVois platform hardware) using sendSerial(). There is no restriction on video modes or frame rates, except as suported by the Camera hardware.
parseSerial(std::string const & str, std::shared_ptr<UserInterface> s) allows the Module to support custom user commands. Engine will forward to this function any command received over Serial or other UserInterface that it does not understand. You should use this for things that go beyond Parameter settings (which is already natively supported by Engine) or built-in commands of Engine (see Command-line interface user guide). For example, one could implement here a command called "start" to allow users to start some specific thing.
supportedCommands(std::ostream & os) should stream out a human-readable description of any custom commands supported by parseSerial(). These will be shown to users when they type "help" over a Serial port.

Note: Every module implementation file should contain a call to JEVOIS_REGISTER_MODULE(MODULENAME) for the module's class. This creates some plain-C entry points that will be used when the module is loaded from a dynamic library (.so) file to instantiate the module. See Tutorial on how to write new machine vision modules for JeVois for examples.

Definition at line 104 of file Module.H.

Inheritance diagram for jevois::Module:

[legend]

Collaboration diagram for jevois::Module:

[legend]

Public Member Functions
	Module (std::string const &instance)
	Constructor. More...

virtual	~Module ()
	Virtual destructor for safe inheritance. More...

virtual void	process (InputFrame &&inframe, OutputFrame &&outframe)
	Processing function, version that receives a frame from camera and sends a frame out over USB. More...

virtual void	process (InputFrame &&inframe)
	Processing function, version that receives a frame from camera and does not use USB. More...

virtual void	process (InputFrame &&inframe, GUIhelper &helper)
	Processing function, version that receives a frame from camera, no USB, but GUI output on JeVois-Pro. More...

virtual void	sendSerial (std::string const &str)
	Send a string over the 'serout' serial port. More...

virtual void	parseSerial (std::string const &str, std::shared_ptr< UserInterface > s)
	Receive a string from a serial port which contains a user command. More...

virtual void	supportedCommands (std::ostream &os)
	Human-readable description of this Module's supported custom commands. More...

Public Member Functions inherited from jevois::Component
	Component (std::string const &instance)
	Constructor. More...

virtual	~Component ()
	Virtual destructor for safe inheritance. More...

template<class Comp , typename... Args>
std::shared_ptr< Comp >	addSubComponent (std::string const &instance, Args &&...args)
	Pseudo-constructor: construct and add another component as a subcomponent of this one. More...

template<class Comp >
void	removeSubComponent (std::shared_ptr< Comp > &component)
	Remove a sub-Component from this Component, by shared_ptr. More...

void	removeSubComponent (std::string const &instance, bool warnIfNotFound=true)
	Remove a sub-Component from this Component, by instance name. More...

template<class Comp = jevois::Component>
std::shared_ptr< Comp >	getSubComponent (std::string const &instance) const
	Get a sub-component by instance name. More...

bool	isTopLevel () const
	Returns true if this component is top-level, i.e., its parent is jevois::Manager. More...

Engine *	engine () const
	Get a handle to our Engine, or throw if we do not have an Engine as root ancestor. More...

bool	initialized () const
	Has this component been initialized yet? More...

const std::string &	className () const
	The class name of this component. More...

const std::string &	instanceName () const
	The instance name of this component. More...

template<typename T >
std::vector< std::string >	setParamVal (std::string const &paramdescriptor, T const &val)
	Set a parameter value. More...

template<typename T >
void	setParamValUnique (std::string const &paramdescriptor, T const &val)
	Set a parameter value, simple version assuming only one parameter match. More...

template<typename T >
std::vector< std::pair< std::string, T > >	getParamVal (std::string const &paramdescriptor) const
	Get parameter(s) value(s) by descriptor. More...

template<typename T >
T	getParamValUnique (std::string const &paramdescriptor) const
	Get a parameter value, simple version assuming only one parameter match. More...

std::vector< std::string >	setParamString (std::string const &paramdescriptor, std::string const &val)
	Set a parameter value, by string. More...

void	setParamStringUnique (std::string const &paramdescriptor, std::string const &val)
	Set a parameter value by string, simple version assuming only one parameter match. More...

std::vector< std::pair< std::string, std::string > >	getParamString (std::string const &paramdescriptor) const
	Get a parameter value, by string. More...

std::string	getParamStringUnique (std::string const &paramdescriptor) const
	Get a parameter value by string, simple version assuming only one parameter match. More...

void	freezeParam (std::string const &paramdescriptor)
	Freeze a parameter, by name, see ParameterBase::freeze() More...

void	unFreezeParam (std::string const &paramdescriptor)
	Unfreeze a parameter, by name, see ParameterBase::unFreeze() More...

void	freezeAllParams ()
	Freeze all parameters. More...

void	unFreezeAllParams ()
	Unfreeze all parameters. More...

std::string	descriptor () const
	Get our full descriptor (including all parents) as [Instancename]:[...]:[...]. More...

void	setParamsFromFile (std::string const &filename)
	Set some parameters from a file. More...

std::istream &	setParamsFromStream (std::istream &is, std::string const &absfile)
	Set some parameters from an open stream. More...

virtual void	paramInfo (std::shared_ptr< UserInterface > s, std::map< std::string, std::string > &categs, bool skipFrozen, std::string const &cname="", std::string const &pfx="")
	Get machine-oriented descriptions of all parameters. More...

void	foreachParam (std::function< void(std::string const &compname, ParameterBase *p)> func, std::string const &cname="")
	Run a function on every param we hold. More...

template<typename T >
std::shared_ptr< DynamicParameter< T > >	addDynamicParameter (std::string const &name, std::string const &description, T const &defaultValue, ParameterCategory const &category)
	Add a new parameter after the Component has already been constructed. More...

template<typename T , template< typename > class ValidValuesSpec>
std::shared_ptr< DynamicParameter< T > >	addDynamicParameter (std::string const &name, std::string const &description, T const &defaultValue, ValidValuesSpec< T > const &validValuesSpec, ParameterCategory const &category)
	Add a new parameter after the Component has already been constructed. More...

template<typename T >
void	setDynamicParameterCallback (std::string const &name, std::function< void(T const &)> cb, bool callnow=true)
	Register a callback with a previously created dynamic parameter. More...

void	removeDynamicParameter (std::string const &name)
	Remove a previously added dynamic parameter. More...

void	setPath (std::string const &path)
	Assign a filesystem path to this component. More...

std::filesystem::path	absolutePath (std::filesystem::path const &path="")
	If given path is relative (not starting with /), prepend the Component path to it. More...

Public Member Functions inherited from jevois::ParameterRegistry
virtual	~ParameterRegistry ()
	Virtual destructor for safe inheritance. More...

Additional Inherited Members
Protected Member Functions inherited from jevois::Component
virtual void	preInit ()
	Called before all sub-Components are init()ed. More...

virtual void	postInit ()
	Called after all sub-Components are init()ed. More...

virtual void	preUninit ()
	Called before all sub-Components are uninit()ed. More...

virtual void	postUninit ()
	Called after all sub-Components are uninit()ed. More...

Protected Member Functions inherited from jevois::ParameterRegistry
void	addParameter (ParameterBase *const param)
	The Parameter class uses this method to register itself on construction with its owning Component. More...

void	removeParameter (ParameterBase *const param)
	The Parameter class uses this method to un-register itself on destruction with its owning Component. More...

void	callbackInitCall ()
	For all parameters that have a callback which has never been called, call it with the default param value. More...

Constructor & Destructor Documentation

◆ Module()

jevois::Module::Module ( std::string const & instance )

Constructor.

the instance is a user-defined string that may be used to differentiate between several instances of the same module.

Definition at line 31 of file Module.C.

◆ ~Module()

jevois::Module::~Module ( )

virtual

Virtual destructor for safe inheritance.

Definition at line 36 of file Module.C.

Member Function Documentation

◆ parseSerial()

void jevois::Module::parseSerial	(	std::string const &	str,
		std::shared_ptr< UserInterface >	s
	)

virtual

Receive a string from a serial port which contains a user command.

This function may be called in between calls to process() with any received string from any of the serial ports. Some commands are parsed upstream already (like "help", set param value, set camera control, etc; see the Engine class) and will not be received here. Only the ones not recognized by the Engine will be received (i.e., custom commands specific to your module).

The default implementation just throws std::runtime_error("Unsupported command"), but some modules may want to override this function to handle custom commands. If you successfully process the command, just return; otherwise, throw, and if your exception derives from std::exception, the Engine will append its what() to the error message issued to the user. When you support commands here, you should update the implementation of supportedCommands() to provide some description of those commands to the users.

The s parameter is the serial port that received the command. You can send any results back to that port using writeString() on it. Note that the Engine will automatically add the 'OK' message upon success, so you do not have to send that here.

Reimplemented in jevois::PythonModule.

Definition at line 63 of file Module.C.

Referenced by jevois::PythonModule::parseSerial().

◆ process() [1/3]

virtual void jevois::Module::process ( InputFrame && inframe )

virtual

Processing function, version that receives a frame from camera and does not use USB.

This function is called once for each grabbed video frame from the camera, and it should complete within the camera's frame period in order to avoid dropping frames. The InputFrame object is a simple wrapper to ensure that the low-level video buffers will always be returned to the low-level camera driver even if the process function throws at any point during the processing. If any error occurs, it is hence ok to throw from within process() at any time, just make sure your locally allocated resources will be freed, which is usually best achieved by using shared_ptr and similar wrappers around them. The Engine (which calls process() on your module for every frame) will catch any exception an proceed to the next frame.

Default implementation in the base class just throws. Derived classes should override it.

Reimplemented in __MODULE__, and jevois::PythonModule.

◆ process() [2/3]

virtual void jevois::Module::process	(	InputFrame &&	inframe,
		GUIhelper &	helper
	)

virtual

Processing function, version that receives a frame from camera, no USB, but GUI output on JeVois-Pro.

This function is called once for each grabbed video frame from the camera, and it should complete within the camera's frame period in order to avoid dropping frames. The InputFrame object is a simple wrapper to ensure that the low-level video buffers will always be returned to the low-level camera driver even if the process function throws at any point during the processing. If any error occurs, it is hence ok to throw from within process() at any time, just make sure your locally allocated resources will be freed, which is usually best achieved by using shared_ptr and similar wrappers around them. The Engine (which calls process() on your module for every frame) will catch any exception an proceed to the next frame.

Default implementation in the base class just throws. Derived classes should override it.

Reimplemented in __MODULE__, and jevois::PythonModule.

◆ process() [3/3]

virtual void jevois::Module::process	(	InputFrame &&	inframe,
		OutputFrame &&	outframe
	)

virtual

Processing function, version that receives a frame from camera and sends a frame out over USB.

This function is called once for each grabbed video frame from the camera, and it should complete within the camera's frame period in order to avoid dropping frames. The InputFrame and OutputFrame objects are simple wrappers to ensure that the low-level video buffers will always be returned to the low-level camera and USB drivers even if the process function throws at any point during the processing. If any error occurs, it is hence ok to throw from within process() at any time, just make sure your locally allocated resources will be freed, which is usually best achieved by using shared_ptr and similar wrappers around them. The Engine (which calls process() on your module for every frame) will catch any exception an proceed to the next frame.

Default implementation in the base class just throws. Derived classes should override it.

Reimplemented in __MODULE__, TutorialEdgeDetectionX4, TutorialEdgeDetection, TutorialConvert, TutorialPassThrough, and jevois::PythonModule.

◆ sendSerial()

void jevois::Module::sendSerial ( std::string const & str )

virtual

Send a string over the 'serout' serial port.

The default implementation just sends the string to the serial port specified by the 'serout' Parameter in Engine (which could be the hardware serial port, the serial-over-USB port, both, or none; see Command-line interface user guide for information about serout). No need to override in most cases. Typically, you would use this function from within process() to send out some results of your processing.

Note that the default 'serout' Parameter setting in Engine is None. This is to allow users to configure parameters, get parameter values, possibly read the help message, etc before the flow of serial outputs from vision processing starts. Once ready to receive serial outputs, one would typically issue a command 'setpar serout Hard' over the JeVois command line to enable serial outputs to the hardware serial port. An Arduino would issue that setpar commands when it is ready to work. See ArduinoTutorial for an example.

Definition at line 54 of file Module.C.

References LFATAL, and jevois::Engine::sendSerial().

Referenced by __MODULE__::process().

◆ supportedCommands()

void jevois::Module::supportedCommands ( std::ostream & os )

virtual

Human-readable description of this Module's supported custom commands.

The format here is free. Just use std::endl to demarcate lines, these will be converted to the appropriate line endings by the serial ports. Default implementation writes "None" to os.

Reimplemented in jevois::PythonModule.

Definition at line 68 of file Module.C.

Referenced by jevois::PythonModule::supportedCommands().

The documentation for this class was generated from the following files:

include/jevois/Core/Module.H
src/jevois/Core/Module.C

Public Member Functions

Additional Inherited Members

Constructor & Destructor Documentation

◆ Module()

◆ ~Module()

Member Function Documentation

◆ parseSerial()

◆ process() [1/3]

◆ process() [2/3]

◆ process() [3/3]

◆ sendSerial()

◆ supportedCommands()