JeVois
1.19
JeVois Smart Embedded Machine Vision Toolkit
|
|
#include <jevois/DNN/PreProcessor.H>
Pre-Processor for neural network pipeline.
This is the first step in a deep neural network processing Pipeline.
Derived classes must implement the pure virtual methods:
They should keep some internal state about what to report, since report() is always called on every frame, but process() may be called less often if the network is slow.
Definition at line 108 of file PreProcessor.H.
Public Member Functions | |
PreProcessor (std::string const &instance) | |
Constructor. More... | |
virtual | ~PreProcessor () |
Destructor. More... | |
virtual void | freeze (bool doit)=0 |
Freeze/unfreeze parameters that users should not change while running. More... | |
std::vector< cv::Mat > | process (jevois::RawImage const &img, std::vector< vsi_nn_tensor_attr_t > const &attrs) |
Extract blobs from input image. More... | |
virtual void | sendreport (jevois::StdModule *mod, jevois::RawImage *outimg=nullptr, jevois::OptGUIhelper *helper=nullptr, bool overlay=true, bool idle=false) |
Report what happened in last process() to console/output video/GUI. More... | |
const cv::Size & | imagesize () const |
Access the last processed image size. More... | |
const std::vector< cv::Mat > & | blobs () const |
Access the last computed blobs (or empty if process() has not yet been called) More... | |
cv::Size | blobsize (size_t num) const |
Access the width and height of a given blob, accounting for NCHW or NHWC. More... | |
void | b2i (float &x, float &y, size_t blobnum=0) |
Convert coordinates from blob back to original image. More... | |
void | b2i (float &x, float &y, cv::Size const &bsiz, bool letterboxed) |
Convert coordinates from blob back to original image, given a known blob size. More... | |
cv::Rect | getUnscaledCropRect (size_t blobnum=0) |
Get unscaled crop rectangle in image coordinates. More... | |
void | getUnscaledCropRect (size_t blobnum, int &tlx, int &tly, int &brx, int &bry) |
Get unscaled crop rectangle in image coordinates. More... | |
void | i2b (float &x, float &y, size_t blobnum=0) |
Convert coordinates from image to blob. More... | |
void | i2b (float &x, float &y, cv::Size const &bsiz, bool letterboxed) |
Convert coordinates from image to blob. More... | |
std::shared_ptr< PreProcessorForPython > | getPreProcForPy () const |
Get a pointer to our python-friendly interface. More... | |
![]() | |
Component (std::string const &instance) | |
Constructor. More... | |
virtual | ~Component () |
Virtual destructor for safe inheritance. More... | |
template<class Comp , typename... Args> | |
std::shared_ptr< Comp > | addSubComponent (std::string const &instance, Args &&...args) |
Pseudo-constructor: construct and add another component as a subcomponent of this one. More... | |
template<class Comp > | |
void | removeSubComponent (std::shared_ptr< Comp > &component) |
Remove a sub-Component from this Component, by shared_ptr. More... | |
void | removeSubComponent (std::string const &instance, bool warnIfNotFound=true) |
Remove a sub-Component from this Component, by instance name. More... | |
template<class Comp = jevois::Component> | |
std::shared_ptr< Comp > | getSubComponent (std::string const &instance) const |
Get a sub-component by instance name. More... | |
bool | isTopLevel () const |
Returns true if this component is top-level, i.e., its parent is jevois::Manager. More... | |
Engine * | engine () const |
Get a handle to our Engine, or throw if we do not have an Engine as root ancestor. More... | |
bool | initialized () const |
Has this component been initialized yet? More... | |
const std::string & | className () const |
The class name of this component. More... | |
const std::string & | instanceName () const |
The instance name of this component. More... | |
template<typename T > | |
std::vector< std::string > | setParamVal (std::string const ¶mdescriptor, T const &val) |
Set a parameter value. More... | |
template<typename T > | |
void | setParamValUnique (std::string const ¶mdescriptor, T const &val) |
Set a parameter value, simple version assuming only one parameter match. More... | |
template<typename T > | |
std::vector< std::pair< std::string, T > > | getParamVal (std::string const ¶mdescriptor) const |
Get parameter(s) value(s) by descriptor. More... | |
template<typename T > | |
T | getParamValUnique (std::string const ¶mdescriptor) const |
Get a parameter value, simple version assuming only one parameter match. More... | |
std::vector< std::string > | setParamString (std::string const ¶mdescriptor, std::string const &val) |
Set a parameter value, by string. More... | |
void | setParamStringUnique (std::string const ¶mdescriptor, std::string const &val) |
Set a parameter value by string, simple version assuming only one parameter match. More... | |
std::vector< std::pair< std::string, std::string > > | getParamString (std::string const ¶mdescriptor) const |
Get a parameter value, by string. More... | |
std::string | getParamStringUnique (std::string const ¶mdescriptor) const |
Get a parameter value by string, simple version assuming only one parameter match. More... | |
void | freezeParam (std::string const ¶mdescriptor) |
Freeze a parameter, by name, see ParameterBase::freeze() More... | |
void | unFreezeParam (std::string const ¶mdescriptor) |
Unfreeze a parameter, by name, see ParameterBase::unFreeze() More... | |
void | freezeAllParams () |
Freeze all parameters. More... | |
void | unFreezeAllParams () |
Unfreeze all parameters. More... | |
std::string | descriptor () const |
Get our full descriptor (including all parents) as [Instancename]:[...]:[...]. More... | |
void | setParamsFromFile (std::string const &filename) |
Set some parameters from a file. More... | |
std::istream & | setParamsFromStream (std::istream &is, std::string const &absfile) |
Set some parameters from an open stream. More... | |
virtual void | paramInfo (std::shared_ptr< UserInterface > s, std::map< std::string, std::string > &categs, bool skipFrozen, std::string const &cname="", std::string const &pfx="") |
Get machine-oriented descriptions of all parameters. More... | |
void | foreachParam (std::function< void(std::string const &compname, ParameterBase *p)> func, std::string const &cname="") |
Run a function on every param we hold. More... | |
template<typename T > | |
std::shared_ptr< DynamicParameter< T > > | addDynamicParameter (std::string const &name, std::string const &description, T const &defaultValue, ParameterCategory const &category) |
Add a new parameter after the Component has already been constructed. More... | |
template<typename T , template< typename > class ValidValuesSpec> | |
std::shared_ptr< DynamicParameter< T > > | addDynamicParameter (std::string const &name, std::string const &description, T const &defaultValue, ValidValuesSpec< T > const &validValuesSpec, ParameterCategory const &category) |
Add a new parameter after the Component has already been constructed. More... | |
template<typename T > | |
void | setDynamicParameterCallback (std::string const &name, std::function< void(T const &)> cb, bool callnow=true) |
Register a callback with a previously created dynamic parameter. More... | |
void | removeDynamicParameter (std::string const &name) |
Remove a previously added dynamic parameter. More... | |
void | setPath (std::string const &path) |
Assign a filesystem path to this component. More... | |
std::filesystem::path | absolutePath (std::filesystem::path const &path="") |
If given path is relative (not starting with /), prepend the Component path to it. More... | |
![]() | |
virtual | ~ParameterRegistry () |
Virtual destructor for safe inheritance. More... | |
Protected Member Functions | |
virtual std::vector< cv::Mat > | process (cv::Mat const &img, bool isrgb, std::vector< vsi_nn_tensor_attr_t > const &attrs, std::vector< cv::Rect > &crops)=0 |
Extract blobs from input image. More... | |
virtual void | report (jevois::StdModule *mod, jevois::RawImage *outimg=nullptr, jevois::OptGUIhelper *helper=nullptr, bool overlay=true, bool idle=false)=0 |
Report what happened in last process() to console/output video/GUI. More... | |
![]() | |
virtual void | preInit () |
Called before all sub-Components are init()ed. More... | |
virtual void | postInit () |
Called after all sub-Components are init()ed. More... | |
virtual void | preUninit () |
Called before all sub-Components are uninit()ed. More... | |
virtual void | postUninit () |
Called after all sub-Components are uninit()ed. More... | |
![]() | |
void | addParameter (ParameterBase *const param) |
The Parameter class uses this method to register itself on construction with its owning Component. More... | |
void | removeParameter (ParameterBase *const param) |
The Parameter class uses this method to un-register itself on destruction with its owning Component. More... | |
void | callbackInitCall () |
For all parameters that have a callback which has never been called, call it with the default param value. More... | |
jevois::dnn::PreProcessor::PreProcessor | ( | std::string const & | instance | ) |
Constructor.
Definition at line 26 of file PreProcessor.C.
|
virtual |
Destructor.
Definition at line 31 of file PreProcessor.C.
void jevois::dnn::PreProcessor::b2i | ( | float & | x, |
float & | y, | ||
cv::Size const & | bsiz, | ||
bool | letterboxed | ||
) |
Convert coordinates from blob back to original image, given a known blob size.
Given coords x,y should be in [0..w-1]x[0..h-1] where w,h are the blob's width and height. This is useful to convert detected boxes back into original input coordinates.
Definition at line 60 of file PreProcessor.C.
References LFATAL.
void jevois::dnn::PreProcessor::b2i | ( | float & | x, |
float & | y, | ||
size_t | blobnum = 0 |
||
) |
Convert coordinates from blob back to original image.
Given coords x,y should be in [0..w-1]x[0..h-1] where w,h are the blob's width and height. This is useful to convert detected boxes back into original input coordinates.
Definition at line 50 of file PreProcessor.C.
References LFATAL.
Referenced by jevois::dnn::PostProcessorDetect::process(), and jevois::dnn::PostProcessorYuNet::process().
const std::vector< cv::Mat > & jevois::dnn::PreProcessor::blobs | ( | ) | const |
Access the last computed blobs (or empty if process() has not yet been called)
Definition at line 35 of file PreProcessor.C.
cv::Size jevois::dnn::PreProcessor::blobsize | ( | size_t | num | ) | const |
Access the width and height of a given blob, accounting for NCHW or NHWC.
Definition at line 43 of file PreProcessor.C.
References jevois::dnn::attrsize(), and LFATAL.
Referenced by jevois::dnn::PostProcessorDetect::process(), and jevois::dnn::PostProcessorYuNet::process().
|
pure virtual |
Freeze/unfreeze parameters that users should not change while running.
Implemented in jevois::dnn::PreProcessorBlob, and jevois::dnn::PreProcessorPython.
std::shared_ptr< jevois::dnn::PreProcessorForPython > jevois::dnn::PreProcessor::getPreProcForPy | ( | ) | const |
Get a pointer to our python-friendly interface.
Definition at line 127 of file PreProcessor.C.
Referenced by jevois::dnn::PostProcessorPythonImpl::process().
void jevois::dnn::PreProcessor::getUnscaledCropRect | ( | size_t | blobnum, |
int & | tlx, | ||
int & | tly, | ||
int & | brx, | ||
int & | bry | ||
) |
Get unscaled crop rectangle in image coordinates.
This is useful to display an image overlay on top of the input image.
Definition at line 88 of file PreProcessor.C.
cv::Rect jevois::dnn::PreProcessor::getUnscaledCropRect | ( | size_t | blobnum = 0 | ) |
Get unscaled crop rectangle in image coordinates.
This is useful to display an image overlay on top of the input image.
Definition at line 81 of file PreProcessor.C.
References LFATAL.
Referenced by jevois::dnn::PostProcessorSegment::process().
void jevois::dnn::PreProcessor::i2b | ( | float & | x, |
float & | y, | ||
cv::Size const & | bsiz, | ||
bool | letterboxed | ||
) |
Convert coordinates from image to blob.
Given coords x,y should be in [0..w-1]x[0..h-1] where w,h are the image's width and height. This is useful to convert mouse coordinates (after they have been converted from screen to image coords) to locations within an input blob.
Definition at line 105 of file PreProcessor.C.
References LFATAL.
void jevois::dnn::PreProcessor::i2b | ( | float & | x, |
float & | y, | ||
size_t | blobnum = 0 |
||
) |
Convert coordinates from image to blob.
Given coords x,y should be in [0..w-1]x[0..h-1] where w,h are the image's width and height. This is useful to convert mouse coordinates (after they have been converted from screen to image coords) to locations within an input blob.
Definition at line 95 of file PreProcessor.C.
References LFATAL.
const cv::Size & jevois::dnn::PreProcessor::imagesize | ( | ) | const |
Access the last processed image size.
Definition at line 39 of file PreProcessor.C.
Referenced by jevois::dnn::PostProcessorDetect::process(), and jevois::dnn::PostProcessorYuNet::process().
|
protectedpure virtual |
Extract blobs from input image.
isrgb should be true if the given img has RGB color order, or false for BGR. Only 3-channel byte images are supported as input.
Implemented in jevois::dnn::PreProcessorBlob, and jevois::dnn::PreProcessorPython.
std::vector< cv::Mat > jevois::dnn::PreProcessor::process | ( | jevois::RawImage const & | img, |
std::vector< vsi_nn_tensor_attr_t > const & | attrs | ||
) |
Extract blobs from input image.
Definition at line 131 of file PreProcessor.C.
References jevois::rawimage::convertToCvBGR(), jevois::rawimage::convertToCvRGB(), jevois::rawimage::cvImage(), jevois::RawImage::fmt, jevois::imu::get(), jevois::RawImage::height, LFATAL, and jevois::RawImage::width.
|
protectedpure virtual |
Report what happened in last process() to console/output video/GUI.
Implemented in jevois::dnn::PreProcessorBlob, and jevois::dnn::PreProcessorPython.
|
virtual |
Report what happened in last process() to console/output video/GUI.
Definition at line 155 of file PreProcessor.C.
References jevois::rawimage::drawRect(), jevois::fccstr(), jevois::imu::get(), jevois::GUIhelper::i2d(), jevois::dnn::shapestr(), and jevois::RawImage::width.