jevois::dnn::Pipeline Class Reference

#include <jevois/DNN/Pipeline.H>

Neural processing pipeline.

A neural processing pipeline in JeVois consists of:

• pre-processing an input image to extract one or more blobs used as input tensors to a deep neural network
• processing the input blobs through a deep network to obtain output blobs
• post-processing the output blobs to display results and send serial messages

A pipeline is typically configured by parsing a YAML config file (zoo file) that determines what kind of pre-processing, network, and post-processing to use, and that sets the parameters for those.

Definition at line 133 of file Pipeline.H.

Inheritance diagram for jevois::dnn::Pipeline:

Collaboration diagram for jevois::dnn::Pipeline:

Public Member Functions
	Pipeline (std::string const &instance)
	Constructor.
virtual	~Pipeline ()
	Destructor.
bool	ready () const
	Returns true when all three of preproc, net, and postproc are ready.
void	process (jevois::RawImage const &inimg, jevois::StdModule *mod, jevois::RawImage *outimg, jevois::OptGUIhelper *helper, bool idle=false)
	Process an input image, send results to serial/image/gui.
void	freeze (bool doit)
	Freeze/unfreeze parameters that users should not change while running.
Public Member Functions inherited from jevois::Component
	Component (std::string const &instance)
	Constructor.
virtual	~Component ()
	Virtual destructor for safe inheritance.
template<class Comp , typename... Args>
std::shared_ptr< Comp >	addSubComponent (std::string const &instance, Args &&...args)
	Pseudo-constructor: construct and add another component as a subcomponent of this one.
template<class Comp >
void	removeSubComponent (std::shared_ptr< Comp > &component)
	Remove a sub-Component from this Component, by shared_ptr.
void	removeSubComponent (std::string const &instance, bool warnIfNotFound=true)
	Remove a sub-Component from this Component, by instance name.
template<class Comp = jevois::Component>
std::shared_ptr< Comp >	getSubComponent (std::string const &instance) const
	Get a sub-component by instance name.
bool	isTopLevel () const
	Returns true if this component is top-level, i.e., its parent is jevois::Manager.
bool	initialized () const
	Has this component been initialized yet?
std::string const &	className () const
	The class name of this component.
std::string const &	instanceName () const
	The instance name of this component.
template<typename T >
std::vector< std::string >	setParamVal (std::string const &paramdescriptor, T const &val)
	Set a parameter value.
template<typename T >
void	setParamValUnique (std::string const &paramdescriptor, T const &val)
	Set a parameter value, simple version assuming only one parameter match.
template<typename T >
std::vector< std::pair< std::string, T > >	getParamVal (std::string const &paramdescriptor) const
	Get parameter(s) value(s) by descriptor.
template<typename T >
T	getParamValUnique (std::string const &paramdescriptor) const
	Get a parameter value, simple version assuming only one parameter match.
std::vector< std::string >	setParamString (std::string const &paramdescriptor, std::string const &val)
	Set a parameter value, by string.
void	setParamStringUnique (std::string const &paramdescriptor, std::string const &val)
	Set a parameter value by string, simple version assuming only one parameter match.
std::vector< std::pair< std::string, std::string > >	getParamString (std::string const &paramdescriptor) const
	Get a parameter value, by string.
std::string	getParamStringUnique (std::string const &paramdescriptor) const
	Get a parameter value by string, simple version assuming only one parameter match.
void	freezeParam (std::string const &paramdescriptor, bool doit)
	Freeze/unfreeze a parameter, by name, see ParameterBase::freeze()
void	freezeAllParams (bool doit)
	Freeze all parameters.
std::string	descriptor () const
	Get our full descriptor (including all parents) as [Instancename]:[...]:[...].
void	setParamsFromFile (std::string const &filename)
	Set some parameters from a file.
std::istream &	setParamsFromStream (std::istream &is, std::string const &absfile)
	Set some parameters from an open stream.
virtual void	paramInfo (std::shared_ptr< UserInterface > s, std::map< std::string, std::string > &categs, bool skipFrozen, std::string const &cname="", std::string const &pfx="")
	Get machine-oriented descriptions of all parameters.
void	foreachParam (std::function< void(std::string const &compname, ParameterBase *p)> func, std::string const &cname="")
	Run a function on every param we hold.
template<typename T >
std::shared_ptr< DynamicParameter< T > >	addDynamicParameter (std::string const &name, std::string const &description, T const &defaultValue, ParameterCategory const &category)
	Add a new parameter after the Component has already been constructed.
template<typename T , template< typename > class ValidValuesSpec>
std::shared_ptr< DynamicParameter< T > >	addDynamicParameter (std::string const &name, std::string const &description, T const &defaultValue, ValidValuesSpec< T > const &validValuesSpec, ParameterCategory const &category)
	Add a new parameter after the Component has already been constructed.
template<typename T >
void	setDynamicParameterCallback (std::string const &name, std::function< void(T const &)> cb, bool callnow=true)
	Register a callback with a previously created dynamic parameter.
void	removeDynamicParameter (std::string const &name, bool throw_if_not_found=true)
	Remove a previously added dynamic parameter.
void	setPath (std::string const &path)
	Assign a filesystem path to this component.
std::filesystem::path	absolutePath (std::filesystem::path const &path="")
	If given path is relative (not starting with /), prepend the Component path to it.
Public Member Functions inherited from jevois::ParameterRegistry
virtual	~ParameterRegistry ()
	Virtual destructor for safe inheritance.

Protected Member Functions
void	postInit () override
	Called after all sub-Components are init()ed.
void	preUninit () override
	Called before all sub-Components are uninit()ed.
void	reloadZoo (std::string const &root, std::string const &filt, std::string const &zoofile)
void	onParamChange (pipeline::zooroot const &param, std::string const &val) override
void	onParamChange (pipeline::zoo const &param, std::string const &val) override
void	onParamChange (pipeline::filter const &param, pipeline::Filter const &val) override
void	onParamChange (pipeline::pipe const &param, std::string const &val) override
void	onParamChange (pipeline::nettype const &param, pipeline::NetType const &val) override
void	onParamChange (pipeline::preproc const &param, pipeline::PreProc const &val) override
void	onParamChange (pipeline::postproc const &param, pipeline::PostProc const &val) override
void	onParamChange (pipeline::benchmark const &param, bool const &val) override
void	showInfo (std::vector< std::string > const &info, jevois::StdModule *mod, jevois::RawImage *outimg, jevois::OptGUIhelper *helper, bool ovl, bool idle)
void	asyncNetWait ()
bool	checkAsyncNetComplete ()
void	showDataPeekWindow (jevois::GUIhelper *helper, bool refresh)
Protected Member Functions inherited from jevois::Component
virtual void	preInit ()
	Called before all sub-Components are init()ed.
virtual void	postUninit ()
	Called after all sub-Components are uninit()ed.
Protected Member Functions inherited from jevois::ParameterRegistry
void	addParameter (ParameterBase *const param)
	The Parameter class uses this method to register itself on construction with its owning Component.
void	removeParameter (ParameterBase *const param)
	The Parameter class uses this method to un-register itself on destruction with its owning Component.
void	callbackInitCall ()
	For all parameters that have a callback which has never been called, call it with the default param value.

Protected Attributes
std::shared_ptr< PreProcessor >	itsPreProcessor
std::shared_ptr< Network >	itsNetwork
std::shared_ptr< PostProcessor >	itsPostProcessor

Related Symbols
(Note that these are not member symbols.)
	JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK (zooroot, std::string, "Path where to find zoo files (.yml). " "If not absolute, it is relative to this module's path", JEVOIS_SHARE_PATH "/dnn", ParamCateg)
	Parameter.
	JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK (zoo, std::string, "Filename for neural network zoo file (.yml). " "If not absolute, it is relative to zooroot", "models.yml", ParamCateg)
	Parameter.
	JEVOIS_DEFINE_ENUM_CLASS (Filter,(All)(OpenCV)(TPU)(NPU)(VPU)(VPUX)(NPUX)(SPU)(ORT))
	Enum.
	JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK (filter, Filter, "Filter to possibly only show as options in " "the 'pipe' parameter some class of models from the zoo", Filter::All, Filter_Values, ParamCateg)
	Parameter.
	JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK (pipe, std::string, "Pipeline to use, which should correspond to a " "top-level entry in the zoo file. Network execution types are: OpenCV: " "on CPU, NPU: on JeVois NPU accelerator, TPU: on Coral Edge TPU if " "available, VPU: on MyriadX VPU if available, NPUX: on NPU via TimVX " "OpenCV extension, VPUX: on CPU via ARM-Compute OpenVino emulation " "of VPU, and SPU: on Hailo8 SPU if available.", "", ParamCateg)
	Parameter.
	JEVOIS_DEFINE_ENUM_CLASS (PreProc,(Blob)(Python))
	Enum.
	JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK (preproc, PreProc, "Pre-Processor to use, usually set automatically " "by selecting a pipeline from the zoo file", PreProc::Blob, PreProc_Values, ParamCateg)
	Parameter.
	JEVOIS_DEFINE_ENUM_CLASS (NetType,(OpenCV)(ORT)(NPU)(TPU)(SPU)(Python))
	Enum.
	JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK (nettype, NetType, "Network runtime framework to use, usually set " "automatically by selecting a pipeline from the zoo file", NetType::OpenCV, NetType_Values, ParamCateg)
	Parameter.
	JEVOIS_DEFINE_ENUM_CLASS (PostProc,(Classify)(Detect)(Segment)(YuNet)(Python)(Stub))
	Enum.
	JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK (postproc, PostProc, "Post-Processor to use, usually set automatically " "by selecting a pipeline from the zoo file", PostProc::Classify, PostProc_Values, ParamCateg)
	Parameter.
	JEVOIS_DEFINE_ENUM_CLASS (Processing,(Sync)(Async))
	Enum.
	JEVOIS_DECLARE_PARAMETER (processing, Processing, "Type of processing: Sync runs pre-processing, " "network, and post-processing sequentially for every frame. Use for fast " "networks only, otherwise it will slow down the GUI... Async runs the network in " "a thread and should be used for networks slower than the camera framerate.", Processing::Async, Processing_Values, ParamCateg)
	Parameter.
	JEVOIS_DECLARE_PARAMETER (overlay, bool, "Show some pipeline info as an overlay over output or GUI video", true, ParamCateg)
	Parameter.
	JEVOIS_DECLARE_PARAMETER (paramwarn, bool, "Issue warning if parameters are specified in the zoo file for " "a pipeline, but the pipeline's component is not using them. Warnings appear " "in the console when the pipeline is loaded.", true, ParamCateg)
	Parameter.
	JEVOIS_DECLARE_PARAMETER (statsfile, std::string, "Append pre/net/post timing statistics in HTML table format " "to the specified file if not empty. If path is relative, it is to " JEVOIS_SHARE_PATH, "", ParamCateg)
	Parameter.
	JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK (benchmark, bool, "Cycle through all networks specified by filter and, " "for each, run it for a while and append pre/net/post timing statistics " "in HTML table format to " JEVOIS_SHARE_PATH "/benchmark.html", false, ParamCateg)
	Parameter.

Constructor & Destructor Documentation

Pipeline()

jevois::dnn::Pipeline::Pipeline

(

std::string const &

instance

)

Constructor.

Definition at line 105 of file Pipeline.C.

References jevois::getNumInstalledNPUs(), jevois::getNumInstalledSPUs(), jevois::getNumInstalledTPUs(), jevois::getNumInstalledVPUs(), and LINFO.

~Pipeline()

jevois::dnn::Pipeline::~Pipeline ( )

virtual

Destructor.

Definition at line 154 of file Pipeline.C.

Member Function Documentation

asyncNetWait()

void jevois::dnn::Pipeline::asyncNetWait ( )

protected

Definition at line 161 of file Pipeline.C.

References LERROR.

checkAsyncNetComplete()

bool jevois::dnn::Pipeline::checkAsyncNetComplete ( )

protected

Definition at line 644 of file Pipeline.C.

freeze()

void jevois::dnn::Pipeline::freeze

(

bool

doit

)

Freeze/unfreeze parameters that users should not change while running.

Definition at line 128 of file Pipeline.C.

onParamChange() [1/8]

void jevois::dnn::Pipeline::onParamChange ( pipeline::benchmark const &  param, bool const &  val  )

overrideprotected

Definition at line 194 of file Pipeline.C.

onParamChange() [2/8]

void jevois::dnn::Pipeline::onParamChange ( pipeline::filter const &  param, pipeline::Filter const &  val  )

overrideprotected

onParamChange() [3/8]

void jevois::dnn::Pipeline::onParamChange ( pipeline::nettype const &  param, pipeline::NetType const &  val  )

overrideprotected

Definition at line 545 of file Pipeline.C.

References LFATAL, and LINFO.

onParamChange() [4/8]

void jevois::dnn::Pipeline::onParamChange ( pipeline::pipe const &  param, std::string const &  val  )

overrideprotected

Definition at line 335 of file Pipeline.C.

References jevois::absolutePath(), jevois::Engine::clearErrors(), LFATAL, and jevois::split().

onParamChange() [5/8]

void jevois::dnn::Pipeline::onParamChange ( pipeline::postproc const &  param, pipeline::PostProc const &  val  )

overrideprotected

Definition at line 605 of file Pipeline.C.

References LINFO.

onParamChange() [6/8]

void jevois::dnn::Pipeline::onParamChange ( pipeline::preproc const &  param, pipeline::PreProc const &  val  )

overrideprotected

Definition at line 525 of file Pipeline.C.

References LINFO.

onParamChange() [7/8]

void jevois::dnn::Pipeline::onParamChange ( pipeline::zoo const &  param, std::string const &  val  )

overrideprotected

Definition at line 209 of file Pipeline.C.

References jevois::absolutePath(), and LINFO.

onParamChange() [8/8]

void jevois::dnn::Pipeline::onParamChange ( pipeline::zooroot const &  param, std::string const &  val  )

overrideprotected

Definition at line 186 of file Pipeline.C.

postInit()

void jevois::dnn::Pipeline::postInit ( )

overrideprotectedvirtual

Called after all sub-Components are init()ed.

Reimplemented from jevois::Component.

Definition at line 140 of file Pipeline.C.

preUninit()

void jevois::dnn::Pipeline::preUninit ( )

overrideprotectedvirtual

Called before all sub-Components are uninit()ed.

Reimplemented from jevois::Component.

Definition at line 147 of file Pipeline.C.

process()

void jevois::dnn::Pipeline::process	(	jevois::RawImage const &	inimg,
		jevois::StdModule *	mod,
		jevois::RawImage *	outimg,
		jevois::OptGUIhelper *	helper,
		bool	idle = false
	)

Process an input image, send results to serial/image/gui.

If the network is not ready, no processing will occur. When helper is not null (i.e., using GUI display), hide the information window when idle is true. This function catches all exceptions and reports them.

Definition at line 659 of file Pipeline.C.

References jevois::absolutePath(), jevois::async(), jevois::GUIhelper::itext(), JEVOIS_SHARE_PATH, jevois::join(), LINFO, jevois::replaceAll(), jevois::GUIhelper::reportAndIgnoreException(), jevois::GUIhelper::reportError(), jevois::secs2str(), jevois::dnn::shapestr(), jevois::split(), jevois::warnAndIgnoreException(), jevois::yuyv::White, and jevois::rawimage::writeText().

ready()

bool jevois::dnn::Pipeline::ready

(

)

const

Returns true when all three of preproc, net, and postproc are ready.

Definition at line 638 of file Pipeline.C.

reloadZoo()

void jevois::dnn::Pipeline::reloadZoo ( std::string const &  root, std::string const &  filt, std::string const &  zoofile  )

protected

showDataPeekWindow()

void jevois::dnn::Pipeline::showDataPeekWindow ( jevois::GUIhelper *  helper, bool  refresh  )

protected

Definition at line 1037 of file Pipeline.C.

References jevois::GUIhelper::combo(), jevois::GUIhelper::reportError(), jevois::dnn::shapestr(), and jevois::GUIhelper::toggleButton().

showInfo()

void jevois::dnn::Pipeline::showInfo ( std::vector< std::string > const &  info, jevois::StdModule *  mod, jevois::RawImage *  outimg, jevois::OptGUIhelper *  helper, bool  ovl, bool  idle  )

protected

Definition at line 1002 of file Pipeline.C.

References jevois::stringStartsWith(), jevois::yuyv::White, and jevois::rawimage::writeText().

Friends And Related Symbol Documentation

JEVOIS_DECLARE_PARAMETER() [1/4]

JEVOIS_DECLARE_PARAMETER ( overlay  , bool  , "Show some pipeline info as an overlay over output or GUI video"  , true  , ParamCateg    )

related

Parameter.

JEVOIS_DECLARE_PARAMETER() [2/4]

JEVOIS_DECLARE_PARAMETER ( paramwarn  , bool  , "Issue warning if parameters are specified in the zoo file for " "a  pipeline, but the pipeline 's component is not using them. Warnings appear " "in the console when the pipeline is loaded."  , true  , ParamCateg    )

related

Parameter.

JEVOIS_DECLARE_PARAMETER() [3/4]

JEVOIS_DECLARE_PARAMETER ( processing  , Processing  , "Type of processing: Sync runs pre-  processing, " "  network, and post-processing sequentially for every frame. Use for fast " "networks  only, otherwise it will slow down the GUI... Async runs the network in " "a thread and should be used for networks slower than the camera framerate."  , Processing::Async  , Processing_Values  , ParamCateg    )

related

Parameter.

JEVOIS_DECLARE_PARAMETER() [4/4]

JEVOIS_DECLARE_PARAMETER ( statsfile  , std::string  , "Append pre/net/post timing statistics in HTML table format " "to the specified file if not empty. If path is  relative, it is to "  JEVOIS_SHARE_PATH, ""  , ParamCateg    )

related

Parameter.

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK() [1/8]

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK ( benchmark  , bool  , "Cycle through all networks specified by filter  and, " "for  each, run it for a while and append pre/net/post timing statistics " "in HTML table format to " JEVOIS_SHARE_PATH "/benchmark.html"  , false  , ParamCateg    )

related

Parameter.

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK() [2/8]

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK ( filter  , Filter  , "Filter to possibly only show as options in " "the 'pipe' parameter some class of models from the zoo"  , Filter::All  , Filter_Values  , ParamCateg    )

related

Parameter.

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK() [3/8]

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK ( nettype  , NetType  , "Network runtime framework to  use, usually set " "automatically by selecting a pipeline from the zoo file"  , NetType::OpenCV  , NetType_Values  , ParamCateg    )

related

Parameter.

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK() [4/8]

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK ( pipe  , std::string  , "Pipeline to  use, which should correspond to a " "top-level entry in the zoo file. Network execution types are:OpenCV:" "on  CPU, NPU:on JeVois NPU  accelerator, TPU:on Coral Edge TPU if " "  available, VPU:on MyriadX VPU if  available, NPUX:on NPU via TimVX " "OpenCV  extension, VPUX:on CPU via ARM-Compute OpenVino emulation " "of  VPU, and SPU:on Hailo8 SPU if available."  , ""  , ParamCateg    )

related

Parameter.

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK() [5/8]

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK ( postproc  , PostProc  , "Post-Processor to  use, usually set automatically " "by selecting a pipeline from the zoo file"  , PostProc::Classify  , PostProc_Values  , ParamCateg    )

related

Parameter.

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK() [6/8]

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK ( preproc  , PreProc  , "Pre-Processor to  use, usually set automatically " "by selecting a pipeline from the zoo file"  , PreProc::Blob  , PreProc_Values  , ParamCateg    )

related

Parameter.

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK() [7/8]

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK ( zoo  , std::string  , "Filename for neural network zoo file (.yml). " "If not  absolute, it is relative to zooroot"  , "models.yml"  , ParamCateg    )

related

Parameter.

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK() [8/8]

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK ( zooroot  , std::string  , "Path where to find zoo files (.yml). " "If not  absolute, it is relative to this module 's path"  , JEVOIS_SHARE_PATH "/dnn"  , ParamCateg    )

related

Parameter.

JEVOIS_DEFINE_ENUM_CLASS() [1/5]

JEVOIS_DEFINE_ENUM_CLASS ( Filter  , (All)(OpenCV)(TPU)(NPU)(VPU)(VPUX)(NPUX)(SPU)(ORT)    )

related

Enum.

JEVOIS_DEFINE_ENUM_CLASS() [2/5]

JEVOIS_DEFINE_ENUM_CLASS ( NetType  , (OpenCV)(ORT)(NPU)(TPU)(SPU)(Python)    )

related

Enum.

JEVOIS_DEFINE_ENUM_CLASS() [3/5]

JEVOIS_DEFINE_ENUM_CLASS ( PostProc  , (Classify)(Detect)(Segment)(YuNet)(Python)(Stub)    )

related

Enum.

JEVOIS_DEFINE_ENUM_CLASS() [4/5]

JEVOIS_DEFINE_ENUM_CLASS ( PreProc  , (Blob)(Python)    )

related

Enum.

JEVOIS_DEFINE_ENUM_CLASS() [5/5]

JEVOIS_DEFINE_ENUM_CLASS ( Processing  , (Sync)(Async)    )

related

Enum.

Member Data Documentation

itsNetwork

std::shared_ptr<Network> jevois::dnn::Pipeline::itsNetwork

protected

Definition at line 162 of file Pipeline.H.

itsPostProcessor

std::shared_ptr<PostProcessor> jevois::dnn::Pipeline::itsPostProcessor

protected

Definition at line 163 of file Pipeline.H.

itsPreProcessor

std::shared_ptr<PreProcessor> jevois::dnn::Pipeline::itsPreProcessor

protected

Definition at line 161 of file Pipeline.H.

---

The documentation for this class was generated from the following files:

• include/jevois/DNN/Pipeline.H
• src/jevois/DNN/Pipeline.C