JeVois
1.19
JeVois Smart Embedded Machine Vision Toolkit
|
|
#include <jevois/DNN/PreProcessorBlob.H>
Pre-Processor for neural network pipeline.
This is the first step in a deep neural network processing Pipeline.
This pre-processor works as follows. As an example, assume a 1024x576 camera input frame and a 224x224 neural network input:
letterbox
is specified, fit the largest possible rectangle, with the aspect ratio of the network input, within the camera frame. For example, 224x224 is square so that would compute a 576x576 square box around the center of the camera frame. Otherwise, use the whole camera frame.letterbox
was off)rgb
parameter and color order in the received camera frame)mean
if not zerostdev
if not 1scale
if not 1. At this point, values will typically be in [0..1] or [-1..1]Because for uint8 (and also signed int8 and dynamic-fixed-point) this leads to nearly a no-op (first transform from native camera range [0..255] to, say, [0..1], then, during quantization, stretch back to [0..255]), fast paths are implemented for these special cases (e.g., uint8 camera input to quantized asymmetric affine uint8 network input). For dynamic fixed point, the fast path uses fast bit-shifting operations; for uint8 asymmetric affine, it is sometimes a no-op.
You can see these steps in the JeVois-Pro GUI (in the window that shows network processing details) by enabling pre-processor parameter details
Definition at line 67 of file PreProcessorBlob.H.
Public Member Functions | |
virtual | ~PreProcessorBlob () |
Destructor. More... | |
void | freeze (bool doit) override |
Freeze/unfreeze parameters that users should not change while running. More... | |
PreProcessor (std::string const &instance) | |
Inherited constructor ok. More... | |
![]() | |
PreProcessor (std::string const &instance) | |
Constructor. More... | |
virtual | ~PreProcessor () |
Destructor. More... | |
std::vector< cv::Mat > | process (jevois::RawImage const &img, std::vector< vsi_nn_tensor_attr_t > const &attrs) |
Extract blobs from input image. More... | |
virtual void | sendreport (jevois::StdModule *mod, jevois::RawImage *outimg=nullptr, jevois::OptGUIhelper *helper=nullptr, bool overlay=true, bool idle=false) |
Report what happened in last process() to console/output video/GUI. More... | |
const cv::Size & | imagesize () const |
Access the last processed image size. More... | |
const std::vector< cv::Mat > & | blobs () const |
Access the last computed blobs (or empty if process() has not yet been called) More... | |
cv::Size | blobsize (size_t num) const |
Access the width and height of a given blob, accounting for NCHW or NHWC. More... | |
void | b2i (float &x, float &y, size_t blobnum=0) |
Convert coordinates from blob back to original image. More... | |
void | b2i (float &x, float &y, cv::Size const &bsiz, bool letterboxed) |
Convert coordinates from blob back to original image, given a known blob size. More... | |
cv::Rect | getUnscaledCropRect (size_t blobnum=0) |
Get unscaled crop rectangle in image coordinates. More... | |
void | getUnscaledCropRect (size_t blobnum, int &tlx, int &tly, int &brx, int &bry) |
Get unscaled crop rectangle in image coordinates. More... | |
void | i2b (float &x, float &y, size_t blobnum=0) |
Convert coordinates from image to blob. More... | |
void | i2b (float &x, float &y, cv::Size const &bsiz, bool letterboxed) |
Convert coordinates from image to blob. More... | |
std::shared_ptr< PreProcessorForPython > | getPreProcForPy () const |
Get a pointer to our python-friendly interface. More... | |
![]() | |
Component (std::string const &instance) | |
Constructor. More... | |
virtual | ~Component () |
Virtual destructor for safe inheritance. More... | |
template<class Comp , typename... Args> | |
std::shared_ptr< Comp > | addSubComponent (std::string const &instance, Args &&...args) |
Pseudo-constructor: construct and add another component as a subcomponent of this one. More... | |
template<class Comp > | |
void | removeSubComponent (std::shared_ptr< Comp > &component) |
Remove a sub-Component from this Component, by shared_ptr. More... | |
void | removeSubComponent (std::string const &instance, bool warnIfNotFound=true) |
Remove a sub-Component from this Component, by instance name. More... | |
template<class Comp = jevois::Component> | |
std::shared_ptr< Comp > | getSubComponent (std::string const &instance) const |
Get a sub-component by instance name. More... | |
bool | isTopLevel () const |
Returns true if this component is top-level, i.e., its parent is jevois::Manager. More... | |
Engine * | engine () const |
Get a handle to our Engine, or throw if we do not have an Engine as root ancestor. More... | |
bool | initialized () const |
Has this component been initialized yet? More... | |
const std::string & | className () const |
The class name of this component. More... | |
const std::string & | instanceName () const |
The instance name of this component. More... | |
template<typename T > | |
std::vector< std::string > | setParamVal (std::string const ¶mdescriptor, T const &val) |
Set a parameter value. More... | |
template<typename T > | |
void | setParamValUnique (std::string const ¶mdescriptor, T const &val) |
Set a parameter value, simple version assuming only one parameter match. More... | |
template<typename T > | |
std::vector< std::pair< std::string, T > > | getParamVal (std::string const ¶mdescriptor) const |
Get parameter(s) value(s) by descriptor. More... | |
template<typename T > | |
T | getParamValUnique (std::string const ¶mdescriptor) const |
Get a parameter value, simple version assuming only one parameter match. More... | |
std::vector< std::string > | setParamString (std::string const ¶mdescriptor, std::string const &val) |
Set a parameter value, by string. More... | |
void | setParamStringUnique (std::string const ¶mdescriptor, std::string const &val) |
Set a parameter value by string, simple version assuming only one parameter match. More... | |
std::vector< std::pair< std::string, std::string > > | getParamString (std::string const ¶mdescriptor) const |
Get a parameter value, by string. More... | |
std::string | getParamStringUnique (std::string const ¶mdescriptor) const |
Get a parameter value by string, simple version assuming only one parameter match. More... | |
void | freezeParam (std::string const ¶mdescriptor) |
Freeze a parameter, by name, see ParameterBase::freeze() More... | |
void | unFreezeParam (std::string const ¶mdescriptor) |
Unfreeze a parameter, by name, see ParameterBase::unFreeze() More... | |
void | freezeAllParams () |
Freeze all parameters. More... | |
void | unFreezeAllParams () |
Unfreeze all parameters. More... | |
std::string | descriptor () const |
Get our full descriptor (including all parents) as [Instancename]:[...]:[...]. More... | |
void | setParamsFromFile (std::string const &filename) |
Set some parameters from a file. More... | |
std::istream & | setParamsFromStream (std::istream &is, std::string const &absfile) |
Set some parameters from an open stream. More... | |
virtual void | paramInfo (std::shared_ptr< UserInterface > s, std::map< std::string, std::string > &categs, bool skipFrozen, std::string const &cname="", std::string const &pfx="") |
Get machine-oriented descriptions of all parameters. More... | |
void | foreachParam (std::function< void(std::string const &compname, ParameterBase *p)> func, std::string const &cname="") |
Run a function on every param we hold. More... | |
template<typename T > | |
std::shared_ptr< DynamicParameter< T > > | addDynamicParameter (std::string const &name, std::string const &description, T const &defaultValue, ParameterCategory const &category) |
Add a new parameter after the Component has already been constructed. More... | |
template<typename T , template< typename > class ValidValuesSpec> | |
std::shared_ptr< DynamicParameter< T > > | addDynamicParameter (std::string const &name, std::string const &description, T const &defaultValue, ValidValuesSpec< T > const &validValuesSpec, ParameterCategory const &category) |
Add a new parameter after the Component has already been constructed. More... | |
template<typename T > | |
void | setDynamicParameterCallback (std::string const &name, std::function< void(T const &)> cb, bool callnow=true) |
Register a callback with a previously created dynamic parameter. More... | |
void | removeDynamicParameter (std::string const &name) |
Remove a previously added dynamic parameter. More... | |
void | setPath (std::string const &path) |
Assign a filesystem path to this component. More... | |
std::filesystem::path | absolutePath (std::filesystem::path const &path="") |
If given path is relative (not starting with /), prepend the Component path to it. More... | |
![]() | |
virtual | ~ParameterRegistry () |
Virtual destructor for safe inheritance. More... | |
Protected Member Functions | |
std::vector< cv::Mat > | process (cv::Mat const &img, bool swaprb, std::vector< vsi_nn_tensor_attr_t > const &attrs, std::vector< cv::Rect > &crops) override |
Extract blobs from input image. More... | |
void | report (jevois::StdModule *mod, jevois::RawImage *outimg=nullptr, jevois::OptGUIhelper *helper=nullptr, bool overlay=true, bool idle=false) override |
Report what happened in last process() to console/output video/GUI. More... | |
![]() | |
virtual void | preInit () |
Called before all sub-Components are init()ed. More... | |
virtual void | postInit () |
Called after all sub-Components are init()ed. More... | |
virtual void | preUninit () |
Called before all sub-Components are uninit()ed. More... | |
virtual void | postUninit () |
Called after all sub-Components are uninit()ed. More... | |
![]() | |
void | addParameter (ParameterBase *const param) |
The Parameter class uses this method to register itself on construction with its owning Component. More... | |
void | removeParameter (ParameterBase *const param) |
The Parameter class uses this method to un-register itself on destruction with its owning Component. More... | |
void | callbackInitCall () |
For all parameters that have a callback which has never been called, call it with the default param value. More... | |
Protected Attributes | |
std::vector< std::string > | itsInfo |
|
virtual |
Destructor.
Definition at line 32 of file PreProcessorBlob.C.
|
overridevirtual |
Freeze/unfreeze parameters that users should not change while running.
Implements jevois::dnn::PreProcessor.
Definition at line 36 of file PreProcessorBlob.C.
jevois::dnn::PreProcessor::PreProcessor |
Inherited constructor ok.
Definition at line 26 of file PreProcessor.C.
|
overrideprotectedvirtual |
Extract blobs from input image.
Implements jevois::dnn::PreProcessor.
Definition at line 42 of file PreProcessorBlob.C.
References jevois::applyLetterBox(), jevois::dnn::attrsize(), jevois::dnn::attrstr(), jevois::cvtypestr(), DETAILS, DETAILS2, jevois::imu::get(), LFATAL, jevois::dnn::quantize(), jevois::dnn::shapestr(), jevois::split(), jevois::to_string(), and jevois::dnn::vsi2cv().
|
overrideprotectedvirtual |
Report what happened in last process() to console/output video/GUI.
Implements jevois::dnn::PreProcessor.
Definition at line 379 of file PreProcessorBlob.C.
|
protected |
Definition at line 90 of file PreProcessorBlob.H.