JeVois  1.16
JeVois Smart Embedded Machine Vision Toolkit
Share this page:
jevois::dnn::Pipeline Class Reference

#include <jevois/DNN/Pipeline.H>

Neural processing pipeline.

A neural processing pipeline in JeVois consists of:

  • pre-processing an input image to extract one or more blobs used as input tensors to a deep neural network
  • processing the input blobs through a deep network to obtain output blobs
  • post-processing the output blobs to display results and send serial messages

A pipeline is typically configured by parsing a YAML config file (zoo file) that determines what kind of pre-processing, network, and post-processing to use, and that sets the parameters for those.

Definition at line 112 of file Pipeline.H.

Inheritance diagram for jevois::dnn::Pipeline:
Collaboration diagram for jevois::dnn::Pipeline:

Public Member Functions

 Pipeline (std::string const &instance)
 Constructor. More...
 
virtual ~Pipeline ()
 Destructor. More...
 
bool ready () const
 Returns true when all three or preproc, net, and postproc are ready. More...
 
void process (jevois::RawImage const &inimg, jevois::StdModule *mod, jevois::RawImage *outimg, jevois::OptGUIhelper *helper, bool idle=false)
 Process an input image, send results to serial/image/gui. More...
 
void freeze (bool doit)
 Freeze/unfreeze parameters that users should not change while running. More...
 
void setCustomPreProcessor (std::shared_ptr< PreProcessor > pp)
 Set a custom pre-processor. More...
 
void setCustomNetwork (std::shared_ptr< Network > n)
 
void setCustomPostProcessor (std::shared_ptr< PostProcessor > pp)
 
- Public Member Functions inherited from jevois::Component
 Component (std::string const &instance)
 Constructor. More...
 
virtual ~Component ()
 Virtual destructor for safe inheritance. More...
 
template<class Comp , typename... Args>
std::shared_ptr< Comp > addSubComponent (std::string const &instance, Args &&...args)
 Pseudo-constructor: construct and add another component as a subcomponent of this one. More...
 
template<class Comp >
void removeSubComponent (std::shared_ptr< Comp > &component)
 Remove a sub-Component from this Component, by shared_ptr. More...
 
void removeSubComponent (std::string const &instance, bool warnIfNotFound=true)
 Remove a sub-Component from this Component, by instance name. More...
 
template<class Comp = jevois::Component>
std::shared_ptr< Comp > getSubComponent (std::string const &instance) const
 Get a sub-component by instance name. More...
 
bool isTopLevel () const
 Returns true if this component is top-level, i.e., its parent is jevois::Manager. More...
 
Engineengine () const
 Get a handle to our Engine, or throw if we do not have an Engine as root ancestor. More...
 
bool initialized () const
 Has this component been initialized yet? More...
 
const std::string & className () const
 The class name of this component. More...
 
const std::string & instanceName () const
 The instance name of this component. More...
 
template<typename T >
std::vector< std::string > setParamVal (std::string const &paramdescriptor, T const &val)
 Set a parameter value. More...
 
template<typename T >
void setParamValUnique (std::string const &paramdescriptor, T const &val)
 Set a parameter value, simple version assuming only one parameter match. More...
 
template<typename T >
std::vector< std::pair< std::string, T > > getParamVal (std::string const &paramdescriptor) const
 Get parameter(s) value(s) by descriptor. More...
 
template<typename T >
getParamValUnique (std::string const &paramdescriptor) const
 Get a parameter value, simple version assuming only one parameter match. More...
 
std::vector< std::string > setParamString (std::string const &paramdescriptor, std::string const &val)
 Set a parameter value, by string. More...
 
void setParamStringUnique (std::string const &paramdescriptor, std::string const &val)
 Set a parameter value by string, simple version assuming only one parameter match. More...
 
std::vector< std::pair< std::string, std::string > > getParamString (std::string const &paramdescriptor) const
 Get a parameter value, by string. More...
 
std::string getParamStringUnique (std::string const &paramdescriptor) const
 Get a parameter value by string, simple version assuming only one parameter match. More...
 
void freezeParam (std::string const &paramdescriptor)
 Freeze a parameter, by name, see ParameterBase::freeze() More...
 
void unFreezeParam (std::string const &paramdescriptor)
 Unfreeze a parameter, by name, see ParameterBase::unFreeze() More...
 
void freezeAllParams ()
 Freeze all parameters. More...
 
void unFreezeAllParams ()
 Unfreeze all parameters. More...
 
std::string descriptor () const
 Get our full descriptor (including all parents) as [Instancename]:[...]:[...]. More...
 
void setParamsFromFile (std::string const &filename)
 Set some parameters from a file. More...
 
std::istream & setParamsFromStream (std::istream &is, std::string const &absfile)
 Set some parameters from an open stream. More...
 
virtual void paramInfo (std::shared_ptr< UserInterface > s, std::map< std::string, std::string > &categs, bool skipFrozen, std::string const &cname="", std::string const &pfx="")
 Get machine-oriented descriptions of all parameters. More...
 
void foreachParam (std::function< void(std::string const &compname, ParameterBase *p)> func, std::string const &cname="")
 Run a function on every param we hold. More...
 
void setPath (std::string const &path)
 Assign a filesystem path to this component. More...
 
std::string absolutePath (std::string const &path="")
 If given path is relative (not starting with /), prepend the Component path to it. More...
 
- Public Member Functions inherited from jevois::ParameterRegistry
virtual ~ParameterRegistry ()
 Virtual destructor for safe inheritance. More...
 

Protected Member Functions

void postInit () override
 Called after all sub-Components are init()ed. More...
 
void preUninit () override
 Called before all sub-Components are uninit()ed. More...
 
- Protected Member Functions inherited from jevois::Component
virtual void preInit ()
 Called before all sub-Components are init()ed. More...
 
virtual void postUninit ()
 Called after all sub-Components are uninit()ed. More...
 
- Protected Member Functions inherited from jevois::ParameterRegistry
void addParameter (ParameterBase *const param)
 The Parameter class uses this method to register itself on construction with its owning Component. More...
 
void removeParameter (ParameterBase *const param)
 The Parameter class uses this method to un-register itself on destruction with its owning Component. More...
 
void callbackInitCall ()
 For all parameters that have a callback which has never been called, call it with the default param value. More...
 

Protected Attributes

std::shared_ptr< PreProcessoritsPreProcessor
 
std::shared_ptr< NetworkitsNetwork
 
std::shared_ptr< PostProcessoritsPostProcessor
 

Constructor & Destructor Documentation

◆ Pipeline()

jevois::dnn::Pipeline::Pipeline ( std::string const &  instance)

◆ ~Pipeline()

jevois::dnn::Pipeline::~Pipeline ( )
virtual

Destructor.

Definition at line 79 of file Pipeline.C.

Member Function Documentation

◆ freeze()

void jevois::dnn::Pipeline::freeze ( bool  doit)

Freeze/unfreeze parameters that users should not change while running.

Definition at line 53 of file Pipeline.C.

◆ postInit()

void jevois::dnn::Pipeline::postInit ( )
overrideprotectedvirtual

Called after all sub-Components are init()ed.

Reimplemented from jevois::Component.

Definition at line 65 of file Pipeline.C.

◆ preUninit()

void jevois::dnn::Pipeline::preUninit ( )
overrideprotectedvirtual

Called before all sub-Components are uninit()ed.

Reimplemented from jevois::Component.

Definition at line 72 of file Pipeline.C.

◆ process()

void jevois::dnn::Pipeline::process ( jevois::RawImage const &  inimg,
jevois::StdModule mod,
jevois::RawImage outimg,
jevois::OptGUIhelper helper,
bool  idle = false 
)

Process an input image, send results to serial/image/gui.

If the network is not ready, no processing will occur. When helper is not null (i.e., using GUI display), hide the information window when idle is true. This function catches all exceptions and reports them.

Definition at line 498 of file Pipeline.C.

References jevois::async(), jevois::imu::get(), jevois::GUIhelper::itext(), jevois::GUIhelper::reportAndIgnoreException(), jevois::secs2str(), jevois::warnAndIgnoreException(), and jevois::rawimage::writeText().

◆ ready()

bool jevois::dnn::Pipeline::ready ( ) const

Returns true when all three or preproc, net, and postproc are ready.

Definition at line 477 of file Pipeline.C.

◆ setCustomNetwork()

void jevois::dnn::Pipeline::setCustomNetwork ( std::shared_ptr< Network n)

See doc of setCustomPreProcessor() for instructions, but use the name "network" instead of "preproc".

Definition at line 423 of file Pipeline.C.

References LINFO.

◆ setCustomPostProcessor()

void jevois::dnn::Pipeline::setCustomPostProcessor ( std::shared_ptr< PostProcessor pp)

See doc of setCustomPreProcessor() for instructions, but use the name "postproc" instead of "preproc".

Definition at line 465 of file Pipeline.C.

References LINFO.

◆ setCustomPreProcessor()

void jevois::dnn::Pipeline::setCustomPreProcessor ( std::shared_ptr< PreProcessor pp)

Set a custom pre-processor.

To create a custom pre-processor, create a new class that derives from jevois::dnn::PreProcessor and that implements the pure virtual functions of that base class. Then, in your module, you should hold a shared_ptr<jevois::Pipeline> which we will call itsPipeline, and in your constructor you instantiate it and add it as a sub-component of your module (see constructor of DNN for an example). Then proceed as follows in your constructor or postInit() or parameter callback:

// Create an instance of our custom pre-processor and attach it as a sub-component of our pipeline
auto sptr = itsPipeline->addSubComponent<MyCustomPreProc>("preproc");
itsPipeline->setCustomPreProcessor(std::static_pointer_cast<jevois::dnn::PreProcessor>(sptr);

And you are ready to rock.

Definition at line 372 of file Pipeline.C.

References LINFO.

Member Data Documentation

◆ itsNetwork

std::shared_ptr<Network> jevois::dnn::Pipeline::itsNetwork
protected

Definition at line 164 of file Pipeline.H.

◆ itsPostProcessor

std::shared_ptr<PostProcessor> jevois::dnn::Pipeline::itsPostProcessor
protected

Definition at line 165 of file Pipeline.H.

◆ itsPreProcessor

std::shared_ptr<PreProcessor> jevois::dnn::Pipeline::itsPreProcessor
protected

Definition at line 163 of file Pipeline.H.


The documentation for this class was generated from the following files: