JeVois
1.20
JeVois Smart Embedded Machine Vision Toolkit
|
|
Classes and utilities to provide abstraction to deep neural networks. Provides interfacing to OpenCV backends (CPU, OpenCL), tensor processing units (TPU) such as Coral Edge TPU and neural processing units (NPU) such as Amlogic A311D NPU.
Classes | |
class | jevois::dnn::Network |
Abstract class to represent a neural network. More... | |
class | jevois::dnn::NetworkHailo |
Wrapper around an DNN neural network running on the Hailo8 neural accelerator. More... | |
class | jevois::dnn::NetworkNPU |
Wrapper around a DNN neural network running on Amlogic A311D NPU accelerator (Verisilicon) More... | |
class | jevois::dnn::NetworkONNX |
Wrapper around an ONNX-Runtime neural network. More... | |
class | jevois::dnn::NetworkOpenCV |
Wrapper around an OpenCV DNN neural network. More... | |
class | jevois::dnn::NetworkPython |
Wrapper around an DNN neural network invoked through python. More... | |
class | jevois::dnn::NetworkTPU |
Wrapper around a Coral TPU neural network. More... | |
class | jevois::dnn::Pipeline |
Neural processing pipeline. More... | |
class | jevois::dnn::PostProcessor |
Post-Processor for neural network pipeline. More... | |
class | jevois::dnn::PostProcessorClassify |
Post-Processor for neural network pipeline. More... | |
class | jevois::dnn::PostProcessorDetect |
Post-Processor for neural network pipeline. More... | |
class | jevois::dnn::PostProcessorPython |
Post-Processor for neural network pipeline. More... | |
class | jevois::dnn::PostProcessorSegment |
Post-Processor for neural network pipeline. More... | |
class | jevois::dnn::PostProcessorStub |
Post-Processor for neural network pipeline. More... | |
class | jevois::dnn::PostProcessorYuNet |
Post-Processor for YuNet face landmarks detector. More... | |
class | jevois::dnn::PreProcessor |
Pre-Processor for neural network pipeline. More... | |
class | jevois::dnn::PreProcessorBlob |
Pre-Processor for neural network pipeline. More... | |
class | jevois::dnn::PreProcessorPython |
Pre-Processor for neural network pipeline written in python. More... | |
Modules | |
DNN-related processors written in python | |
In addition to writing DNN pre/net/post processors in C++, JeVois supports writing them in Python. | |
Functions | |
std::map< int, std::string > | jevois::dnn::readLabelsFile (std::string const &fname) |
Read a label file. More... | |
std::string | jevois::dnn::getLabel (std::map< int, std::string > const &labels, int id) |
Get a label from an id. More... | |
int | jevois::dnn::stringToRGBA (std::string const &label, unsigned char alpha=128) |
Compute a color from a label name. More... | |
void | jevois::dnn::topK (float const *pfProb, float *pfMaxProb, uint32_t *pMaxClass, uint32_t outputCount, uint32_t topNum) |
Get top-k entries and their indices. More... | |
std::string | jevois::dnn::shapestr (cv::Mat const &m) |
Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional cv::Mat with data type TYPE. More... | |
std::string | jevois::dnn::shapestr (TfLiteTensor const *t) |
Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional TfLiteTensor with data type TYPE. More... | |
std::string | jevois::dnn::shapestr (vsi_nn_tensor_attr_t const &attr) |
Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional NPU tensor with data type TYPE. More... | |
std::vector< size_t > | jevois::dnn::strshape (std::string const &str) |
Get a vector of size_t from a string containing AxBxC... More... | |
int | jevois::dnn::tf2cv (TfLiteType t) |
Convert from TensorFlow data type to OpenCV. More... | |
vsi_nn_type_e | jevois::dnn::tf2vsi (TfLiteType t) |
Convert from TensorFlow data type to vsi_nn. More... | |
int | jevois::dnn::vsi2cv (vsi_nn_type_e t) |
Convert from NPU data type to OpenCV. More... | |
void | jevois::dnn::clamp (cv::Rect &r, int width, int height) |
Clamp a rectangle to within given image width and height. More... | |
void | jevois::dnn::clamp (cv::Rect2f &r, float width, float height) |
Clamp a rectangle to within given image width and height. More... | |
std::vector< vsi_nn_tensor_attr_t > | jevois::dnn::parseTensorSpecs (std::string const &specs) |
Parse tensor specification. More... | |
cv::Mat | jevois::dnn::attrmat (vsi_nn_tensor_attr_t const &attr, void *dataptr=nullptr) |
Construct a cv::Mat from attr and possibly data pointer. More... | |
std::vector< int > | jevois::dnn::attrdims (vsi_nn_tensor_attr_t const &attr) |
Get a tensor dims as a vector of int, useful to construct a matching cv::Mat. More... | |
cv::Size | jevois::dnn::attrsize (vsi_nn_tensor_attr_t const &attr) |
Get a tensor's (width, height) size in cv::Size format, skipping over other dimensions. More... | |
std::string | jevois::dnn::attrstr (vsi_nn_tensor_attr_t const &attr) |
Get a string describing the specs of a tensor, including quantification specs (not provided by shapestr()) More... | |
bool | jevois::dnn::attrmatch (vsi_nn_tensor_attr_t const &attr, cv::Mat const &blob) |
Check that a cv::Mat blob matches exactly the spec of an attr. More... | |
vsi_nn_tensor_attr_t | jevois::dnn::tensorattr (TfLiteTensor const *t) |
Get tensor shape and type attributes for a TensorFlow Lite tensor. More... | |
size_t | jevois::dnn::softmax (float const *input, size_t const n, size_t const stride, float const fac, float *output, bool maxonly) |
Apply softmax to a float vector. More... | |
cv::Mat | jevois::dnn::quantize (cv::Mat const &m, vsi_nn_tensor_attr_t const &attr) |
Quantize from float32 to fixed-point according to the quantization spec in attr. More... | |
cv::Mat | jevois::dnn::dequantize (cv::Mat const &m, vsi_nn_tensor_attr_t const &attr) |
Dequantize an output to float32 according to the quantization spec in attr. More... | |
size_t | jevois::dnn::effectiveDims (cv::Mat const &m) |
Returns the number of non-unit dims in a cv::Mat. More... | |
std::string | jevois::dnn::shapestr (hailo_vstream_info_t const &vi) |
Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional Hailo tensor with data type TYPE. More... | |
vsi_nn_tensor_attr_t | jevois::dnn::tensorattr (hailo_vstream_info_t const &vi) |
Get tensor shape and type attributes for a Hailo tensor. More... | |
vsi_nn_type_e | jevois::dnn::hailo2vsi (hailo_format_type_t t) |
Convert from Hailo data type to vsi_nn. More... | |
vsi_nn_type_e | jevois::dnn::onnx2vsi (ONNXTensorElementDataType t) |
Convert from ONNX-Runtime data type to vsi_nn. More... | |
std::string | jevois::dnn::shapestr (Ort::ConstTensorTypeAndShapeInfo const &ti) |
Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional ONNX tensor with data type TYPE. More... | |
vsi_nn_tensor_attr_t | jevois::dnn::tensorattr (Ort::ConstTensorTypeAndShapeInfo const &ti) |
Get tensor shape and type attributes for an ONNX-runtime tensor. More... | |
std::vector< int > jevois::dnn::attrdims | ( | vsi_nn_tensor_attr_t const & | attr | ) |
Get a tensor dims as a vector of int, useful to construct a matching cv::Mat.
Definition at line 477 of file Utils.C.
Referenced by jevois::dnn::attrmat(), jevois::dnn::NetworkHailo::load(), jevois::dnn::Network::process(), and jevois::dnn::quantize().
cv::Mat jevois::dnn::attrmat | ( | vsi_nn_tensor_attr_t const & | attr, |
void * | dataptr = nullptr |
||
) |
Construct a cv::Mat from attr and possibly data pointer.
If dataptr is nullptr, new memory will be allocated for the cv::Mat. Caller must ensure data outlives the cv::Mat, and is responsible for eventually de-allocating the data. Usually, with non-null dataptr, this is only to be used as a temporary re-casting, e.g., to recast a received tensor into a Mat before dequantizing it, then forgetting about that Mat.
Definition at line 470 of file Utils.C.
References jevois::dnn::attrdims(), and jevois::dnn::vsi2cv().
Referenced by jevois::dnn::NetworkONNX::doprocess(), jevois::dnn::NetworkNPU::doprocess(), jevois::dnn::NetworkHailo::load(), and jevois::dnn::Network::process().
bool jevois::dnn::attrmatch | ( | vsi_nn_tensor_attr_t const & | attr, |
cv::Mat const & | blob | ||
) |
Check that a cv::Mat blob matches exactly the spec of an attr.
Definition at line 703 of file Utils.C.
References jevois::dnn::vsi2cv().
Referenced by jevois::dnn::dequantize(), jevois::dnn::NetworkHailo::doprocess(), and jevois::dnn::NetworkNPU::doprocess().
cv::Size jevois::dnn::attrsize | ( | vsi_nn_tensor_attr_t const & | attr | ) |
Get a tensor's (width, height) size in cv::Size format, skipping over other dimensions.
Definition at line 486 of file Utils.C.
References jevois::dnn::attrstr().
Referenced by jevois::dnn::PreProcessor::blobsize(), and jevois::dnn::PreProcessorBlob::process().
std::string jevois::dnn::attrstr | ( | vsi_nn_tensor_attr_t const & | attr | ) |
Get a string describing the specs of a tensor, including quantification specs (not provided by shapestr())
Definition at line 511 of file Utils.C.
References jevois::to_string().
Referenced by jevois::dnn::attrsize(), jevois::dnn::NetworkONNX::doprocess(), jevois::dnn::NetworkHailo::doprocess(), jevois::dnn::NetworkNPU::doprocess(), jevois::dnn::NetworkTPU::inputShapes(), jevois::dnn::NetworkHailo::load(), jevois::dnn::NetworkTPU::outputShapes(), jevois::dnn::PreProcessorPythonImpl::process(), and jevois::dnn::PreProcessorBlob::process().
void jevois::dnn::clamp | ( | cv::Rect & | r, |
int | width, | ||
int | height | ||
) |
Clamp a rectangle to within given image width and height.
Definition at line 366 of file Utils.C.
Referenced by jevois::dnn::PostProcessorDetect::process(), and jevois::dnn::PostProcessorYuNet::process().
void jevois::dnn::clamp | ( | cv::Rect2f & | r, |
float | width, | ||
float | height | ||
) |
cv::Mat jevois::dnn::dequantize | ( | cv::Mat const & | m, |
vsi_nn_tensor_attr_t const & | attr | ||
) |
Dequantize an output to float32 according to the quantization spec in attr.
attr should have the type and quantization details of m, returned tensor is float32
Definition at line 792 of file Utils.C.
References jevois::dnn::attrmatch(), LFATAL, and jevois::dnn::shapestr().
Referenced by jevois::dnn::NetworkHailo::doprocess(), and jevois::dnn::NetworkNPU::doprocess().
size_t jevois::dnn::effectiveDims | ( | cv::Mat const & | m | ) |
std::string jevois::dnn::getLabel | ( | std::map< int, std::string > const & | labels, |
int | id | ||
) |
Get a label from an id.
If no entry is found in the map, return the id as a string.
Definition at line 67 of file Utils.C.
References jevois::to_string().
Referenced by jevois::dnn::PostProcessorClassify::process(), and jevois::dnn::PostProcessorDetect::process().
vsi_nn_type_e jevois::dnn::hailo2vsi | ( | hailo_format_type_t | t | ) |
Convert from Hailo data type to vsi_nn.
Definition at line 352 of file Utils.C.
References jevois::to_string().
Referenced by jevois::dnn::tensorattr().
vsi_nn_type_e jevois::dnn::onnx2vsi | ( | ONNXTensorElementDataType | t | ) |
Convert from ONNX-Runtime data type to vsi_nn.
Definition at line 207 of file Utils.C.
References jevois::to_string().
Referenced by jevois::dnn::tensorattr().
std::vector< vsi_nn_tensor_attr_t > jevois::dnn::parseTensorSpecs | ( | std::string const & | specs | ) |
Parse tensor specification.
If the specification is empty, an empty vector is returned. Throws std::range_error on any parsing error.
Definition at line 386 of file Utils.C.
References jevois::split(), and jevois::dnn::strshape().
Referenced by jevois::dnn::NetworkPython::inputShapes(), jevois::dnn::NetworkTPU::inputShapes(), jevois::dnn::NetworkOpenCV::inputShapes(), jevois::dnn::NetworkNPU::inputShapes(), jevois::dnn::NetworkNPU::load(), jevois::dnn::Network::onParamChange(), jevois::dnn::NetworkPython::outputShapes(), jevois::dnn::NetworkOpenCV::outputShapes(), jevois::dnn::NetworkTPU::outputShapes(), and jevois::dnn::NetworkNPU::outputShapes().
cv::Mat jevois::dnn::quantize | ( | cv::Mat const & | m, |
vsi_nn_tensor_attr_t const & | attr | ||
) |
Quantize from float32 to fixed-point according to the quantization spec in attr.
m should be float32, typically normalized to [0..1[ or [-1..1[ already. attr is the desired quantized type and method (DFP, AA, etc)
Definition at line 717 of file Utils.C.
References jevois::dnn::attrdims(), LFATAL, jevois::dnn::shapestr(), and jevois::dnn::vsi2cv().
Referenced by jevois::dnn::PreProcessorBlob::process().
std::map< int, std::string > jevois::dnn::readLabelsFile | ( | std::string const & | fname | ) |
Read a label file.
Two formats are allowed: one class name per line, or one class number followed by one class name per file.
Definition at line 24 of file Utils.C.
References LERROR, LFATAL, LINFO, and jevois::replaceStringAll().
Referenced by jevois::dnn::PostProcessorClassify::onParamChange().
std::string jevois::dnn::shapestr | ( | cv::Mat const & | m | ) |
Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional cv::Mat with data type TYPE.
Definition at line 104 of file Utils.C.
References jevois::cvtypestr(), and jevois::to_string().
Referenced by jevois::dnn::dequantize(), jevois::dnn::NetworkTPU::doprocess(), jevois::dnn::NetworkONNX::doprocess(), jevois::dnn::NetworkNPU::doprocess(), jevois::dnn::NetworkHailo::doprocess(), jevois::GUIhelperPython::drawPoly2(), jevois::dnn::NetworkTPU::load(), jevois::dnn::NetworkONNX::load(), jevois::dnn::PostProcessorSegment::process(), jevois::dnn::PostProcessorDetect::process(), jevois::dnn::PreProcessorBlob::process(), jevois::dnn::Pipeline::process(), jevois::dnn::Network::process(), jevois::dnn::quantize(), jevois::dnn::PreProcessor::sendreport(), jevois::dnn::shapestr(), and jevois::dnn::PostProcessorDetectYOLO::yolo().
std::string jevois::dnn::shapestr | ( | hailo_vstream_info_t const & | vi | ) |
Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional Hailo tensor with data type TYPE.
Definition at line 251 of file Utils.C.
References jevois::dnn::shapestr(), and jevois::dnn::tensorattr().
std::string jevois::dnn::shapestr | ( | Ort::ConstTensorTypeAndShapeInfo const & | ti | ) |
Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional ONNX tensor with data type TYPE.
Definition at line 173 of file Utils.C.
References jevois::to_string().
std::string jevois::dnn::shapestr | ( | TfLiteTensor const * | t | ) |
Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional TfLiteTensor with data type TYPE.
Definition at line 114 of file Utils.C.
References jevois::to_string().
std::string jevois::dnn::shapestr | ( | vsi_nn_tensor_attr_t const & | attr | ) |
Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional NPU tensor with data type TYPE.
Definition at line 143 of file Utils.C.
References jevois::to_string().
size_t jevois::dnn::softmax | ( | float const * | input, |
size_t const | n, | ||
size_t const | stride, | ||
float const | fac, | ||
float * | output, | ||
bool | maxonly | ||
) |
Apply softmax to a float vector.
n is the number of elements to process, stride is the increment in the arrays from one element to the next. So the arrays should have size n * stride. Returns the index in [0..n*stride[ of the highest scoring element. If maxonly is true, only output[returned index] is valid.
Definition at line 667 of file Utils.C.
References LFATAL.
Referenced by jevois::dnn::PostProcessorClassify::process().
int jevois::dnn::stringToRGBA | ( | std::string const & | label, |
unsigned char | alpha = 128 |
||
) |
Compute a color from a label name.
Definition at line 75 of file Utils.C.
Referenced by jevois::dnn::PostProcessorDetect::report().
std::vector< size_t > jevois::dnn::strshape | ( | std::string const & | str | ) |
Get a vector of size_t from a string containing AxBxC...
Definition at line 276 of file Utils.C.
References jevois::split().
Referenced by jevois::dnn::parseTensorSpecs(), and jevois::dnn::Network::process().
vsi_nn_tensor_attr_t jevois::dnn::tensorattr | ( | hailo_vstream_info_t const & | vi | ) |
Get tensor shape and type attributes for a Hailo tensor.
Definition at line 601 of file Utils.C.
References jevois::dnn::hailo2vsi(), and jevois::to_string().
vsi_nn_tensor_attr_t jevois::dnn::tensorattr | ( | Ort::ConstTensorTypeAndShapeInfo const & | ti | ) |
Get tensor shape and type attributes for an ONNX-runtime tensor.
Definition at line 233 of file Utils.C.
References jevois::dnn::onnx2vsi().
vsi_nn_tensor_attr_t jevois::dnn::tensorattr | ( | TfLiteTensor const * | t | ) |
Get tensor shape and type attributes for a TensorFlow Lite tensor.
Definition at line 562 of file Utils.C.
References LFATAL, and jevois::dnn::tf2vsi().
Referenced by jevois::dnn::NetworkTPU::inputShapes(), jevois::dnn::NetworkONNX::load(), jevois::dnn::NetworkHailo::load(), jevois::dnn::NetworkTPU::outputShapes(), and jevois::dnn::shapestr().
int jevois::dnn::tf2cv | ( | TfLiteType | t | ) |
Convert from TensorFlow data type to OpenCV.
Definition at line 285 of file Utils.C.
Referenced by jevois::dnn::NetworkTPU::doprocess().
vsi_nn_type_e jevois::dnn::tf2vsi | ( | TfLiteType | t | ) |
Convert from TensorFlow data type to vsi_nn.
Definition at line 329 of file Utils.C.
Referenced by jevois::dnn::tensorattr().
void jevois::dnn::topK | ( | float const * | pfProb, |
float * | pfMaxProb, | ||
uint32_t * | pMaxClass, | ||
uint32_t | outputCount, | ||
uint32_t | topNum | ||
) |
Get top-k entries and their indices.
Definition at line 84 of file Utils.C.
Referenced by jevois::dnn::PostProcessorClassify::process().
int jevois::dnn::vsi2cv | ( | vsi_nn_type_e | t | ) |
Convert from NPU data type to OpenCV.
Definition at line 307 of file Utils.C.
References jevois::to_string().
Referenced by jevois::dnn::attrmat(), jevois::dnn::attrmatch(), jevois::dnn::NetworkONNX::doprocess(), jevois::dnn::PreProcessorBlob::process(), and jevois::dnn::quantize().