JeVois  1.5
JeVois Smart Embedded Machine Vision Toolkit
Share this page:
jevois::Module Class Reference

#include <jevois/Core/Module.H>

Virtual base class for a vision processing module.

Module is the base class to implement camera-to-USB frame-by-frame video processing. The Engine instantiates one class derived from Module, according to the current VideoMapping selected by the end user (e.g., current image resolution, format, and frame rate setected by a webcam viewing program on a host computer). The Module is loaded as shared object (.so) file according to the VideoMapping definitions in videomappings.cfg and the current VideoMapping selected by the user.

Module derives from Component and as such can contain:

  • any number of Parameter settings
  • any arbitrarily complex sub-hierarchy of Component objects to implement various functionality. Parameter settings from all the sub-components are available to the Module and to users connected over Serial ports of the Engine.

This allows one to implement complex vision processing pipelines efficiently and with substantial code re-use. For example, one may first want to implement an EdgeDetector or Saliency component, with Parameter settings for various thresholds, features, etc. One can then create any number of top-level objects that derive from Module and that may contain one or more EdgeDetector, Saliency, etc components in their hierarchy of sub-components, with the implementation in the module simply routing images from one Component to another to create a processing pipeline.

Classes that derive from Module should implement up to four functions:

  • process(InputFrame && inframe, OutputFrame && outframe) is called once per iteration of the Engine main loop when the current VideoMapping specifies both a particular Camera resolution and format, and a USB resolution and format. This function should process the received input frame and fill the pixel buffer of the output frame with results. Memory has already been allocated for both the input and output images before process() is called. Because the output image is actually allocated by the USB Gadget driver (and, ultimately, by the Linux kernel), its pixel memory location cannot be moved (hence, do not attempt to copy the output image or replace it by another image, etc; just write pixel data into the output image's pixel array). There is no restriction on video modes or frame rates, except as suported by the Camera hardware, and as limited by USB bandwidth. For most implementations, matching the input and output frame rate is easiest, and means that each invocation of process() would access and use both of the provided InputFrame and OutputFrame (one-input-to-one-output processing pipeline). But this is not mandatory. For example, a motion flow computation Module for use in a flying drone may have 320x240 YUYV 53.0fps inputs and 100x142 GREY 10.6fps output (since output frame rate is 5x lower than input in this example, the process() function would here get, fill, and send the OutputFrame only once every 5 times it is called; implementation of the process() function should keep track of that, e.g., through a member variable that gets incremented each time process() is called). In addition to filling the pixel data of the OutputFrame, process() may also send results over the serial ports (e.g., for use by an Arduino connected to the JeVois platform hardware) using sendSerial().
  • process(InputFrame && inframe) is called once per Camera frame when the current VideoMapping specifies a particular Camera resolution and format, and NONE as USB output format. This function should process the received input frame and would typicaly then send results to serial ports (e.g., for use by an Arduino connected to the JeVois platform hardware) using sendSerial(). There is no restriction on video modes or frame rates, except as suported by the Camera hardware.
  • parseSerial(std::string const & str, std::shared_ptr<UserInterface> s) allows the Module to support custom user commands. Engine will forward to this function any command received over Serial or other UserInterface that it does not understand. You should use this for things that go beyond Parameter settings (which is already natively supported by Engine) or built-in commands of Engine (see Command-line interface user guide). For example, one could implement here a command called "start" to allow users to start some specific thing.
  • supportedCommands(std::ostream & os) should stream out a human-readable description of any custom commands supported by parseSerial(). These will be shown to users when they type "help" over a Serial port.
Note
Every module implementation file should contain a call to JEVOIS_REGISTER_MODULE(MODULENAME) for the module's class. This creates some plain-C entry points that will be used when the module is loaded from a dynamic library (.so) file to instantiate the module. See Tutorial on how to write new machine vision modules for JeVois for examples.

Definition at line 279 of file Module.H.

Inheritance diagram for jevois::Module:
Collaboration diagram for jevois::Module:

Public Member Functions

 Module (std::string const &instance)
 Constructor. More...
 
virtual ~Module ()
 Virtual destructor for safe inheritance. More...
 
virtual void process (InputFrame &&inframe, OutputFrame &&outframe)
 Processing function, version that receives a frame from camera and sends a frame out over USB. More...
 
virtual void process (InputFrame &&inframe)
 Processing function, version that receives a frame from camera and does not use USB. More...
 
virtual void sendSerial (std::string const &str)
 Send a string over the 'serout' serial port. More...
 
virtual void parseSerial (std::string const &str, std::shared_ptr< UserInterface > s)
 Receive a string from a serial port which contains a user command. More...
 
virtual void supportedCommands (std::ostream &os)
 Human-readable description of this Module's supported custom commands. More...
 
- Public Member Functions inherited from jevois::Component
 Component (std::string const &instance)
 Constructor. More...
 
virtual ~Component ()
 Virtual destructor for safe inheritance. More...
 
template<>
std::shared_ptr< ComponentgetSubComponent (std::string const &instance) const
 
template<class Comp , typename... Args>
std::shared_ptr< Comp > addSubComponent (std::string const &instance, Args &&...args)
 Pseudo-constructor: construct and add another component as a subcomponent of this one. More...
 
template<class Comp >
void removeSubComponent (std::shared_ptr< Comp > &component)
 Remove a sub-Component from this Component, by shared_ptr. More...
 
void removeSubComponent (std::string const &instance, bool warnIfNotFound=true)
 Remove a sub-Component from this Component, by instance name. More...
 
template<class Comp = jevois::Component>
std::shared_ptr< Comp > getSubComponent (std::string const &instance) const
 Get a sub-component by instance name. More...
 
bool isTopLevel () const
 Returns true if this component is top-level, i.e., its parent is jevois::Manager. More...
 
bool initialized () const
 Has this component been initialized yet? More...
 
std::string const & className () const
 The class name of this component. More...
 
std::string const & instanceName () const
 The instance name of this component. More...
 
template<typename T >
std::vector< std::string > setParamVal (std::string const &paramdescriptor, T const &val)
 Set a parameter value. More...
 
template<typename T >
void setParamValUnique (std::string const &paramdescriptor, T const &val)
 Set a parameter value, simple version assuming only one parameter match. More...
 
template<typename T >
std::vector< std::pair< std::string, T > > getParamVal (std::string const &paramdescriptor) const
 Get parameter(s) value(s) by descriptor. More...
 
template<typename T >
getParamValUnique (std::string const &paramdescriptor) const
 Get a parameter value, simple version assuming only one parameter match. More...
 
std::vector< std::string > setParamString (std::string const &paramdescriptor, std::string const &val)
 Set a parameter value, by string. More...
 
void setParamStringUnique (std::string const &paramdescriptor, std::string const &val)
 Set a parameter value by string, simple version assuming only one parameter match. More...
 
std::vector< std::pair< std::string, std::string > > getParamString (std::string const &paramdescriptor) const
 Get a parameter value, by string. More...
 
std::string getParamStringUnique (std::string const &paramdescriptor) const
 Get a parameter value by string, simple version assuming only one parameter match. More...
 
void freezeParam (std::string const &paramdescriptor)
 Freeze a parameter, by name, see ParameterBase::freeze() More...
 
void unFreezeParam (std::string const &paramdescriptor)
 Unfreeze a parameter, by name, see ParameterBase::unFreeze() More...
 
void freezeAllParams ()
 Freeze all parameters. More...
 
void unFreezeAllParams ()
 Unfreeze all parameters. More...
 
std::string descriptor () const
 Get our full descriptor (including all parents) as [Instancename]:[...]:[...]. More...
 
void setParamsFromFile (std::string const &filename)
 Set some parameters from a file. More...
 
std::istream & setParamsFromStream (std::istream &is, std::string const &absfile)
 Set some parameters from an open stream. More...
 
void setPath (std::string const &path)
 Assign a filesystem path to this component. More...
 
std::string absolutePath (std::string const &path="")
 If given path is relative (not starting with /), prepend the Component path to it. More...
 
- Public Member Functions inherited from jevois::ParameterRegistry
virtual ~ParameterRegistry ()
 Virtual destructor for safe inheritance. More...
 

Related Functions

(Note that these are not member functions.)

 JEVOIS_DEFINE_ENUM_CLASS (SerStyle,(Terse)(Normal)(Detail)(Fine)) JEVOIS_DECLARE_PARAMETER(serstyle
 Enum for Parameter. More...
 
 JEVOIS_DECLARE_PARAMETER (serprec, unsigned int, "Number of decimal points in standardized serial messages as " "defined in http://jevois.org/doc/UserSerialStyle.html", 0U, ParamCateg)
 Parameter. More...
 

Additional Inherited Members

- Protected Member Functions inherited from jevois::Component
virtual void preInit ()
 Called before all sub-Components are init()ed. More...
 
virtual void postInit ()
 Called after all sub-Components are init()ed. More...
 
virtual void preUninit ()
 Called before all sub-Components are uninit()ed. More...
 
virtual void postUninit ()
 Called after all sub-Components are uninit()ed. More...
 
- Protected Member Functions inherited from jevois::ParameterRegistry
void addParameter (ParameterBase *const param)
 The Parameter class uses this method to register itself on construction with its owning Component. More...
 
void removeParameter (ParameterBase *const param)
 The Parameter class uses this method to un-register itself on destruction with its owning Component. More...
 
void callbackInitCall ()
 For all parameters that have a callback which has never been called, call it with the default param value. More...
 

Constructor & Destructor Documentation

◆ Module()

jevois::Module::Module ( std::string const &  instance)

Constructor.

the instance is a user-defined string that may be used to differentiate between several instances of the same module.

Definition at line 181 of file Module.C.

◆ ~Module()

jevois::Module::~Module ( )
virtual

Virtual destructor for safe inheritance.

Definition at line 186 of file Module.C.

References LFATAL, and process().

Member Function Documentation

◆ parseSerial()

void jevois::Module::parseSerial ( std::string const &  str,
std::shared_ptr< UserInterface s 
)
virtual

Receive a string from a serial port which contains a user command.

This function may be called in between calls to process() with any received string from any of the serial ports. Some commands are parsed upstream already (like "help", set param value, set camera control, etc; see the Engine class) and will not be received here. Only the ones not recognized by the Engine will be received (i.e., custom commands specific to your module).

The default implementation just throws std::runtime_error("Unsupported command"), but some modules may want to override this function to handle custom commands. If you successfully process the command, just return; otherwise, throw, and if your exception derives from std::exception, the Engine will append its what() to the error message issued to the user. When you support commands here, you should update the implementation of supportedCommands() to provide some description of those commands to the users.

The s parameter is the serial port that received the command. You can send any results back to that port using writeString() on it. Note that the Engine will automatically add the 'OK' message upon success, so you do not have to send that here.

Reimplemented in jevois::PythonModule.

Definition at line 207 of file Module.C.

Referenced by jevois::PythonModule::parseSerial().

◆ process() [1/2]

virtual void jevois::Module::process ( InputFrame &&  inframe,
OutputFrame &&  outframe 
)
virtual

Processing function, version that receives a frame from camera and sends a frame out over USB.

This function is called once for each grabbed video frame from the camera, and it should complete within the camera's frame period in order to avoid dropping frames. The InputFrame and OutputFrame objects are simple wrappers to ensure that the low-level video buffers will always be returned to the low-level camera and USB drivers even if the process function throws at any point during the processing. If any error occurs, it is hence ok to throw from within process() at any time, just make sure your locally allocated resources will be freed, which is usually best achieved by using shared_ptr and similar wrappers around them. The Engine (which calls process() on your module for every frame) will catch any exception an proceed to the next frame.

Default implementation in the base class just throws. Derived classes should override it.

Reimplemented in jevois::PythonModule, TutorialEdgeDetectionX4, TutorialEdgeDetection, TutorialConvert, and TutorialPassThrough.

Referenced by ~Module().

◆ process() [2/2]

virtual void jevois::Module::process ( InputFrame &&  inframe)
virtual

Processing function, version that receives a frame from camera and does not use USB.

This function is called once for each grabbed video frame from the camera, and it should complete within the camera's frame period in order to avoid dropping frames. The InputFrame object is a simple wrapper to ensure that the low-level video buffers will always be returned to the low-level camera driver even if the process function throws at any point during the processing. If any error occurs, it is hence ok to throw from within process() at any time, just make sure your locally allocated resources will be freed, which is usually best achieved by using shared_ptr and similar wrappers around them. The Engine (which calls process() on your module for every frame) will catch any exception an proceed to the next frame.

Default implementation in the base class just throws. Derived classes should override it.

Reimplemented in jevois::PythonModule.

◆ sendSerial()

void jevois::Module::sendSerial ( std::string const &  str)
virtual

Send a string over the 'serout' serial port.

The default implementation just sends the string to the serial port specified by the 'serout' Parameter in Engine (which could be the hardware serial port, the serial-over-USB port, both, or none; see Command-line interface user guide for information about serout). No need to override in most cases. Typically, you would use this function from within process() to send out some results of your processing.

Note that the default 'serout' Parameter setting in Engine is None. This is to allow users to configure parameters, get parameter values, possibly read the help message, etc before the flow of serial outputs from vision processing starts. Once ready to receive serial outputs, one would typically issue a command 'setpar serout Hard' over the JeVois command line to enable serial outputs to the hardware serial port. An Arduino would issue that setpar commands when it is ready to work. See ArduinoTutorial for an example.

Definition at line 198 of file Module.C.

References LFATAL, and jevois::Engine::sendSerial().

Referenced by jevois::StdModule::sendSerialContour2D(), jevois::StdModule::sendSerialStd1Dx(), jevois::StdModule::sendSerialStd1Dy(), jevois::StdModule::sendSerialStd2D(), and jevois::StdModule::sendSerialStd3D().

◆ supportedCommands()

void jevois::Module::supportedCommands ( std::ostream &  os)
virtual

Human-readable description of this Module's supported custom commands.

The format here is free. Just use std::endl to demarcate lines, these will be converted to the appropriate line endings by the serial ports. Default implementation writes "None" to os.

Reimplemented in jevois::PythonModule.

Definition at line 212 of file Module.C.

Referenced by jevois::PythonModule::supportedCommands().

Friends And Related Function Documentation

◆ JEVOIS_DECLARE_PARAMETER()

JEVOIS_DECLARE_PARAMETER ( serprec  ,
unsigned  int,
"Number of decimal points in standardized serial messages as " "defined in http://jevois.org/doc/UserSerialStyle.html"  ,
0U  ,
ParamCateg   
)
related

◆ JEVOIS_DEFINE_ENUM_CLASS()

JEVOIS_DEFINE_ENUM_CLASS ( SerStyle  ,
(Terse)(Normal)(Detail)(Fine)   
)
related

Enum for Parameter.

Parameter


The documentation for this class was generated from the following files: