JeVois  1.16
JeVois Smart Embedded Machine Vision Toolkit
Share this page:
jevois::dnn::PreProcessor Class Referenceabstract

#include <jevois/DNN/PreProcessor.H>

Pre-Processor for neural network pipeline.

This is the first step in a deep neural network processing Pipeline.

Derived classes must implement the pure virtual methods:

  • process(cv::Mat const & img, bool isrgb): process and input image and generate some tensors (blobs)
  • sendreport(): describe what process() did to humans
  • freeze(): freeze/unfreeze parameters that users should not change at runtime

They should keep some internal state about what to report, since sendreport() is always called on every frame, but process() may be called less often if the network is slow.

Definition at line 76 of file PreProcessor.H.

Inheritance diagram for jevois::dnn::PreProcessor:
Collaboration diagram for jevois::dnn::PreProcessor:

Public Member Functions

virtual ~PreProcessor ()
 Destructor. More...
 
virtual void freeze (bool doit)=0
 Freeze/unfreeze parameters that users should not change while running. More...
 
std::vector< cv::Mat > process (jevois::RawImage const &img, std::vector< vsi_nn_tensor_attr_t > const &attrs)
 Extract blobs from input image. More...
 
virtual void sendreport (jevois::StdModule *mod, jevois::RawImage *outimg=nullptr, jevois::OptGUIhelper *helper=nullptr, bool overlay=true, bool idle=false)
 Report what happened in last process() to console/output video/GUI. More...
 
const cv::Size & imagesize () const
 Access the last processed image size. More...
 
const std::vector< cv::Mat > & blobs () const
 Access the last computed blobs (or empty if process() has not yet been called) More...
 
cv::Size blobsize (size_t num) const
 Access the width and height of a given blob, accounting for NCHW or NHWC. More...
 
void b2i (float &x, float &y, size_t blobnum=0)
 Convert coordinates from blob back to original image. More...
 
void b2i (float &x, float &y, cv::Size const &bsiz, bool letterboxed)
 Convert coordinates from blob back to original image, given a known blob size. More...
 
cv::Rect getUnscaledCropRect (size_t blobnum=0)
 Get unscaled crop rectangle in image coordinates. More...
 
void getUnscaledCropRect (size_t blobnum, int &tlx, int &tly, int &brx, int &bry)
 Get unscaled crop rectangle in image coordinates. More...
 
- Public Member Functions inherited from jevois::Component
 Component (std::string const &instance)
 Constructor. More...
 
virtual ~Component ()
 Virtual destructor for safe inheritance. More...
 
template<class Comp , typename... Args>
std::shared_ptr< Comp > addSubComponent (std::string const &instance, Args &&...args)
 Pseudo-constructor: construct and add another component as a subcomponent of this one. More...
 
template<class Comp >
void removeSubComponent (std::shared_ptr< Comp > &component)
 Remove a sub-Component from this Component, by shared_ptr. More...
 
void removeSubComponent (std::string const &instance, bool warnIfNotFound=true)
 Remove a sub-Component from this Component, by instance name. More...
 
template<class Comp = jevois::Component>
std::shared_ptr< Comp > getSubComponent (std::string const &instance) const
 Get a sub-component by instance name. More...
 
bool isTopLevel () const
 Returns true if this component is top-level, i.e., its parent is jevois::Manager. More...
 
Engineengine () const
 Get a handle to our Engine, or throw if we do not have an Engine as root ancestor. More...
 
bool initialized () const
 Has this component been initialized yet? More...
 
const std::string & className () const
 The class name of this component. More...
 
const std::string & instanceName () const
 The instance name of this component. More...
 
template<typename T >
std::vector< std::string > setParamVal (std::string const &paramdescriptor, T const &val)
 Set a parameter value. More...
 
template<typename T >
void setParamValUnique (std::string const &paramdescriptor, T const &val)
 Set a parameter value, simple version assuming only one parameter match. More...
 
template<typename T >
std::vector< std::pair< std::string, T > > getParamVal (std::string const &paramdescriptor) const
 Get parameter(s) value(s) by descriptor. More...
 
template<typename T >
getParamValUnique (std::string const &paramdescriptor) const
 Get a parameter value, simple version assuming only one parameter match. More...
 
std::vector< std::string > setParamString (std::string const &paramdescriptor, std::string const &val)
 Set a parameter value, by string. More...
 
void setParamStringUnique (std::string const &paramdescriptor, std::string const &val)
 Set a parameter value by string, simple version assuming only one parameter match. More...
 
std::vector< std::pair< std::string, std::string > > getParamString (std::string const &paramdescriptor) const
 Get a parameter value, by string. More...
 
std::string getParamStringUnique (std::string const &paramdescriptor) const
 Get a parameter value by string, simple version assuming only one parameter match. More...
 
void freezeParam (std::string const &paramdescriptor)
 Freeze a parameter, by name, see ParameterBase::freeze() More...
 
void unFreezeParam (std::string const &paramdescriptor)
 Unfreeze a parameter, by name, see ParameterBase::unFreeze() More...
 
void freezeAllParams ()
 Freeze all parameters. More...
 
void unFreezeAllParams ()
 Unfreeze all parameters. More...
 
std::string descriptor () const
 Get our full descriptor (including all parents) as [Instancename]:[...]:[...]. More...
 
void setParamsFromFile (std::string const &filename)
 Set some parameters from a file. More...
 
std::istream & setParamsFromStream (std::istream &is, std::string const &absfile)
 Set some parameters from an open stream. More...
 
virtual void paramInfo (std::shared_ptr< UserInterface > s, std::map< std::string, std::string > &categs, bool skipFrozen, std::string const &cname="", std::string const &pfx="")
 Get machine-oriented descriptions of all parameters. More...
 
void foreachParam (std::function< void(std::string const &compname, ParameterBase *p)> func, std::string const &cname="")
 Run a function on every param we hold. More...
 
void setPath (std::string const &path)
 Assign a filesystem path to this component. More...
 
std::string absolutePath (std::string const &path="")
 If given path is relative (not starting with /), prepend the Component path to it. More...
 
- Public Member Functions inherited from jevois::ParameterRegistry
virtual ~ParameterRegistry ()
 Virtual destructor for safe inheritance. More...
 

Protected Member Functions

virtual std::vector< cv::Mat > process (cv::Mat const &img, bool isrgb, std::vector< vsi_nn_tensor_attr_t > const &attrs, std::vector< cv::Rect > &crops)=0
 Extract blobs from input image. More...
 
virtual void report (jevois::StdModule *mod, jevois::RawImage *outimg=nullptr, jevois::OptGUIhelper *helper=nullptr, bool overlay=true, bool idle=false)=0
 Report what happened in last process() to console/output video/GUI. More...
 
- Protected Member Functions inherited from jevois::Component
virtual void preInit ()
 Called before all sub-Components are init()ed. More...
 
virtual void postInit ()
 Called after all sub-Components are init()ed. More...
 
virtual void preUninit ()
 Called before all sub-Components are uninit()ed. More...
 
virtual void postUninit ()
 Called after all sub-Components are uninit()ed. More...
 
- Protected Member Functions inherited from jevois::ParameterRegistry
void addParameter (ParameterBase *const param)
 The Parameter class uses this method to register itself on construction with its owning Component. More...
 
void removeParameter (ParameterBase *const param)
 The Parameter class uses this method to un-register itself on destruction with its owning Component. More...
 
void callbackInitCall ()
 For all parameters that have a callback which has never been called, call it with the default param value. More...
 

Constructor & Destructor Documentation

◆ ~PreProcessor()

jevois::dnn::PreProcessor::~PreProcessor ( )
virtual

Destructor.

Definition at line 25 of file PreProcessor.C.

Member Function Documentation

◆ b2i() [1/2]

void jevois::dnn::PreProcessor::b2i ( float &  x,
float &  y,
cv::Size const &  bsiz,
bool  letterboxed 
)

Convert coordinates from blob back to original image, given a known blob size.

Given coords x,y should be in [0..w-1]x[0..h-1] where w,h are the blob's width and height. This is useful to convert detected boxes back into original input coordinates.

Definition at line 54 of file PreProcessor.C.

References LFATAL.

◆ b2i() [2/2]

void jevois::dnn::PreProcessor::b2i ( float &  x,
float &  y,
size_t  blobnum = 0 
)

Convert coordinates from blob back to original image.

Given coords x,y should be in [0..w-1]x[0..h-1] where w,h are the blob's width and height. This is useful to convert detected boxes back into original input coordinates.

Definition at line 44 of file PreProcessor.C.

References LFATAL.

Referenced by jevois::dnn::PostProcessorDetect::process().

◆ blobs()

const std::vector< cv::Mat > & jevois::dnn::PreProcessor::blobs ( ) const

Access the last computed blobs (or empty if process() has not yet been called)

Definition at line 29 of file PreProcessor.C.

◆ blobsize()

cv::Size jevois::dnn::PreProcessor::blobsize ( size_t  num) const

Access the width and height of a given blob, accounting for NCHW or NHWC.

Definition at line 37 of file PreProcessor.C.

References jevois::dnn::attrsize(), and LFATAL.

Referenced by jevois::dnn::PostProcessorDetect::process().

◆ freeze()

virtual void jevois::dnn::PreProcessor::freeze ( bool  doit)
pure virtual

Freeze/unfreeze parameters that users should not change while running.

Implemented in jevois::dnn::PreProcessorBlob.

◆ getUnscaledCropRect() [1/2]

void jevois::dnn::PreProcessor::getUnscaledCropRect ( size_t  blobnum,
int &  tlx,
int &  tly,
int &  brx,
int &  bry 
)

Get unscaled crop rectangle in image coordinates.

This is useful to display an image overlay on top of the input image.

Definition at line 82 of file PreProcessor.C.

◆ getUnscaledCropRect() [2/2]

cv::Rect jevois::dnn::PreProcessor::getUnscaledCropRect ( size_t  blobnum = 0)

Get unscaled crop rectangle in image coordinates.

This is useful to display an image overlay on top of the input image.

Definition at line 75 of file PreProcessor.C.

References LFATAL.

Referenced by jevois::dnn::PostProcessorSegment::process().

◆ imagesize()

const cv::Size & jevois::dnn::PreProcessor::imagesize ( ) const

Access the last processed image size.

Definition at line 33 of file PreProcessor.C.

Referenced by jevois::dnn::PostProcessorDetect::process().

◆ process() [1/2]

virtual std::vector<cv::Mat> jevois::dnn::PreProcessor::process ( cv::Mat const &  img,
bool  isrgb,
std::vector< vsi_nn_tensor_attr_t > const &  attrs,
std::vector< cv::Rect > &  crops 
)
protectedpure virtual

Extract blobs from input image.

isrgb should be true if the given img has RGB color order, or false for BGR. Only 3-channel byte images are supported as input.

Implemented in jevois::dnn::PreProcessorBlob.

◆ process() [2/2]

std::vector< cv::Mat > jevois::dnn::PreProcessor::process ( jevois::RawImage const &  img,
std::vector< vsi_nn_tensor_attr_t > const &  attrs 
)

◆ report()

virtual void jevois::dnn::PreProcessor::report ( jevois::StdModule mod,
jevois::RawImage outimg = nullptr,
jevois::OptGUIhelper helper = nullptr,
bool  overlay = true,
bool  idle = false 
)
protectedpure virtual

Report what happened in last process() to console/output video/GUI.

Implemented in jevois::dnn::PreProcessorBlob.

◆ sendreport()

void jevois::dnn::PreProcessor::sendreport ( jevois::StdModule mod,
jevois::RawImage outimg = nullptr,
jevois::OptGUIhelper helper = nullptr,
bool  overlay = true,
bool  idle = false 
)
virtual

Report what happened in last process() to console/output video/GUI.

Definition at line 113 of file PreProcessor.C.

References jevois::rawimage::drawRect(), jevois::fccstr(), jevois::imu::get(), jevois::GUIhelper::i2d(), jevois::dnn::shapestr(), and jevois::RawImage::width.


The documentation for this class was generated from the following files: