JeVois  1.16
JeVois Smart Embedded Machine Vision Toolkit
Share this page:
Tensor/Neural Processing networks

Classes and utilities to provide abstraction to deep neural networks. Provides interfacing to OpenCV backends (CPU, OpenCL), tensor processing units (TPU) such as Coral Edge TPU and neural processing units (NPU) such as Amlogic A311D NPU. Supported on JeVois-Pro only.

Collaboration diagram for Tensor/Neural Processing networks:

Classes

class  jevois::dnn::Network
 Abstract class to represent a neural network. More...
 
class  jevois::dnn::NetworkOpenCV
 Wrapper around an OpenCV DNN neural network. More...
 
class  jevois::dnn::NetworkTPU
 Wrapper around a Coral TPU neural network. More...
 
class  jevois::dnn::Pipeline
 Neural processing pipeline. More...
 
class  jevois::dnn::PostProcessor
 Post-Processor for neural network pipeline. More...
 
class  jevois::dnn::PostProcessorClassify
 Post-Processor for neural network pipeline. More...
 
class  jevois::dnn::PostProcessorDetect
 Post-Processor for neural network pipeline. More...
 
class  jevois::dnn::PostProcessorSegment
 Post-Processor for neural network pipeline. More...
 
class  jevois::dnn::PreProcessor
 Pre-Processor for neural network pipeline. More...
 
class  jevois::dnn::PreProcessorBlob
 Pre-Processor for neural network pipeline. More...
 

Functions

std::map< int, std::string > jevois::dnn::readLabelsFile (std::string const &fname)
 Read a label file. More...
 
std::string jevois::dnn::getLabel (std::map< int, std::string > const &labels, int id)
 Get a label from an id. More...
 
int jevois::dnn::stringToRGBA (std::string const &label, unsigned char alpha=128)
 Compute a color from a label name. More...
 
void jevois::dnn::topK (float const *pfProb, float *pfMaxProb, uint32_t *pMaxClass, uint32_t outputCount, uint32_t topNum)
 Get top-k entries and their indices. More...
 
std::string jevois::dnn::shapestr (cv::Mat const &m)
 Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional cv::Mat with data type TYPE. More...
 
std::string jevois::dnn::shapestr (TfLiteTensor const *t)
 Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional TfLiteTensor with data type TYPE. More...
 
std::string jevois::dnn::shapestr (vsi_nn_tensor_attr_t const &attr)
 Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional NPU tensor with data type TYPE. More...
 
std::vector< size_t > jevois::dnn::strshape (std::string const &str)
 Get a vector of size_t from a string containing AxBxC... More...
 
int jevois::dnn::tf2cv (TfLiteType t)
 Convert from TensorFlow data type to OpenCV. More...
 
vsi_nn_type_e jevois::dnn::tf2vsi (TfLiteType t)
 Convert from TensorFlow data type to vsi_nn. More...
 
int jevois::dnn::vsi2cv (vsi_nn_type_e t)
 Convert from NPU data type to OpenCV. More...
 
void jevois::dnn::clamp (cv::Rect &r, int width, int height)
 Clamp a rectangle to within given image width and height. More...
 
std::vector< vsi_nn_tensor_attr_t > jevois::dnn::parseTensorSpecs (std::string const &specs)
 Parse tensor specification. More...
 
cv::Size jevois::dnn::attrsize (vsi_nn_tensor_attr_t const &attr)
 Get a tensor's size in cv::Size format. More...
 
std::string jevois::dnn::attrstr (vsi_nn_tensor_attr_t const &attr)
 Get a string describing the specs of a tensor. More...
 
vsi_nn_tensor_attr_t jevois::dnn::tensorattr (TfLiteTensor const *t)
 Get tensor shape and type attributes for a TensorFlow Lite tensor. More...
 
void jevois::dnn::softmax (float const *input, size_t n, float fac, float *output)
 Apply softmax to a float vector. More...
 

Function Documentation

◆ attrsize()

cv::Size jevois::dnn::attrsize ( vsi_nn_tensor_attr_t const &  attr)

Get a tensor's size in cv::Size format.

Definition at line 344 of file Utils.C.

References jevois::dnn::attrstr().

Referenced by jevois::dnn::PreProcessor::blobsize(), and jevois::dnn::PreProcessorBlob::process().

◆ attrstr()

std::string jevois::dnn::attrstr ( vsi_nn_tensor_attr_t const &  attr)

Get a string describing the specs of a tensor.

Definition at line 360 of file Utils.C.

References jevois::to_string().

Referenced by jevois::dnn::attrsize().

◆ clamp()

void jevois::dnn::clamp ( cv::Rect &  r,
int  width,
int  height 
)

Clamp a rectangle to within given image width and height.

Definition at line 250 of file Utils.C.

Referenced by jevois::dnn::PostProcessorDetect::process().

◆ getLabel()

std::string jevois::dnn::getLabel ( std::map< int, std::string > const &  labels,
int  id 
)

Get a label from an id.

If no entry is found in the map, return the id as a string.

Definition at line 68 of file Utils.C.

References jevois::to_string().

Referenced by jevois::dnn::PostProcessorClassify::process(), and jevois::dnn::PostProcessorDetect::process().

◆ parseTensorSpecs()

std::vector< vsi_nn_tensor_attr_t > jevois::dnn::parseTensorSpecs ( std::string const &  specs)

Parse tensor specification.

If the specification is empty, an empty vector is returned. Throws std::range_error on any parsing error.

Definition at line 260 of file Utils.C.

References jevois::split(), and jevois::dnn::strshape().

Referenced by jevois::dnn::NetworkOpenCV::inputShapes(), jevois::dnn::NetworkTPU::inputShapes(), jevois::dnn::NetworkTPU::outputShapes(), and jevois::dnn::NetworkOpenCV::outputShapes().

◆ readLabelsFile()

std::map< int, std::string > jevois::dnn::readLabelsFile ( std::string const &  fname)

Read a label file.

Two formats are allowed: one class name per line, or one class number followed by one class name per file.

Definition at line 25 of file Utils.C.

References LERROR, LFATAL, and LINFO.

Referenced by jevois::dnn::PostProcessorClassify::onParamChange().

◆ shapestr() [1/3]

std::string jevois::dnn::shapestr ( cv::Mat const &  m)

◆ shapestr() [2/3]

std::string jevois::dnn::shapestr ( TfLiteTensor const *  t)

Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional TfLiteTensor with data type TYPE.

Definition at line 115 of file Utils.C.

References jevois::to_string().

◆ shapestr() [3/3]

std::string jevois::dnn::shapestr ( vsi_nn_tensor_attr_t const &  attr)

Get a string of the form: "nD AxBxC... TYPE" from an n-dimensional NPU tensor with data type TYPE.

Definition at line 144 of file Utils.C.

References jevois::to_string().

◆ softmax()

void jevois::dnn::softmax ( float const *  input,
size_t  n,
float  fac,
float *  output 
)

Apply softmax to a float vector.

Definition at line 443 of file Utils.C.

Referenced by jevois::dnn::PostProcessorClassify::process(), and jevois::dnn::npu::yolo().

◆ stringToRGBA()

int jevois::dnn::stringToRGBA ( std::string const &  label,
unsigned char  alpha = 128 
)

Compute a color from a label name.

Definition at line 76 of file Utils.C.

Referenced by jevois::dnn::PostProcessorDetect::report().

◆ strshape()

std::vector< size_t > jevois::dnn::strshape ( std::string const &  str)

Get a vector of size_t from a string containing AxBxC...

Definition at line 173 of file Utils.C.

References jevois::split().

Referenced by jevois::dnn::parseTensorSpecs().

◆ tensorattr()

vsi_nn_tensor_attr_t jevois::dnn::tensorattr ( TfLiteTensor const *  t)

Get tensor shape and type attributes for a TensorFlow Lite tensor.

Definition at line 410 of file Utils.C.

References LFATAL, and jevois::dnn::tf2vsi().

◆ tf2cv()

int jevois::dnn::tf2cv ( TfLiteType  t)

Convert from TensorFlow data type to OpenCV.

Definition at line 182 of file Utils.C.

References LFATAL.

Referenced by jevois::dnn::NetworkTPU::doprocess().

◆ tf2vsi()

vsi_nn_type_e jevois::dnn::tf2vsi ( TfLiteType  t)

Convert from TensorFlow data type to vsi_nn.

Definition at line 227 of file Utils.C.

References LFATAL.

Referenced by jevois::dnn::tensorattr().

◆ topK()

void jevois::dnn::topK ( float const *  pfProb,
float *  pfMaxProb,
uint32_t *  pMaxClass,
uint32_t  outputCount,
uint32_t  topNum 
)

Get top-k entries and their indices.

Definition at line 85 of file Utils.C.

Referenced by jevois::dnn::PostProcessorClassify::process().

◆ vsi2cv()

int jevois::dnn::vsi2cv ( vsi_nn_type_e  t)

Convert from NPU data type to OpenCV.

Definition at line 205 of file Utils.C.

References jevois::to_string().

Referenced by jevois::dnn::PreProcessorBlob::process().