JeVoisBase  1.5
JeVois Smart Embedded Machine Vision Toolkit Base Modules
Share this page:
ObjectRecognitionBase Class Referenceabstract

Abstract base class for an object recognition component. More...

#include <jevoisbase/Components/ObjectRecognition/ObjectRecognition.H>

Inheritance diagram for ObjectRecognitionBase:
Collaboration diagram for ObjectRecognitionBase:

Public Types

typedef std::vector< tiny_dnn::float_t, tiny_dnn::aligned_allocator< tiny_dnn::float_t, 64 > > vec_t
 Type used by tiny-dnn for the results: More...
 

Public Member Functions

 ObjectRecognitionBase (std::string const &instance)
 Constructor. More...
 
virtual ~ObjectRecognitionBase ()
 Virtual destructor for safe inheritance. More...
 
virtual void define ()=0
 Define the network structure. More...
 
virtual tiny_dnn::index3d< size_t > insize () const =0
 Get the input size for the current network, useful to prepare inputs to process() More...
 
virtual void train (std::string const &path)=0
 Train the network. More...
 
virtual vec_t process (cv::Mat const &img, bool normalize=true)=0
 Process an image, results are confidence for each category. More...
 
virtual std::string const & category (size_t idx) const =0
 Return the name of a given category (0-based index in the vector of results) More...
 
- Public Member Functions inherited from jevois::Component
 Component (std::string const &instance)
 
virtual ~Component ()
 
std::shared_ptr< ComponentgetSubComponent (std::string const &instance) const
 
std::shared_ptr< Comp > addSubComponent (std::string const &instance, Args &&...args)
 
void removeSubComponent (std::shared_ptr< Comp > &component)
 
void removeSubComponent (std::string const &instance, bool warnIfNotFound=true)
 
std::shared_ptr< Comp > getSubComponent (std::string const &instance) const
 
bool isTopLevel () const
 
bool initialized () const
 
std::string const & className () const
 
std::string const & instanceName () const
 
std::vector< std::string > setParamVal (std::string const &paramdescriptor, T const &val)
 
void setParamValUnique (std::string const &paramdescriptor, T const &val)
 
std::vector< std::pair< std::string, T > > getParamVal (std::string const &paramdescriptor) const
 
getParamValUnique (std::string const &paramdescriptor) const
 
std::vector< std::string > setParamString (std::string const &paramdescriptor, std::string const &val)
 
void setParamStringUnique (std::string const &paramdescriptor, std::string const &val)
 
std::vector< std::pair< std::string, std::string > > getParamString (std::string const &paramdescriptor) const
 
std::string getParamStringUnique (std::string const &paramdescriptor) const
 
void freezeParam (std::string const &paramdescriptor)
 
void unFreezeParam (std::string const &paramdescriptor)
 
void freezeAllParams ()
 
void unFreezeAllParams ()
 
std::string descriptor () const
 
void setParamsFromFile (std::string const &filename)
 
std::istream & setParamsFromStream (std::istream &is, std::string const &absfile)
 
void setPath (std::string const &path)
 
std::string absolutePath (std::string const &path="")
 
std::shared_ptr< Comp > addSubComponent (std::string const &instance, Args &&...args)
 
void removeSubComponent (std::shared_ptr< Comp > &component)
 
void removeSubComponent (std::string const &instance, bool warnIfNotFound=true)
 
std::shared_ptr< Comp > getSubComponent (std::string const &instance) const
 
bool isTopLevel () const
 
bool initialized () const
 
std::string const & className () const
 
std::string const & instanceName () const
 
std::vector< std::string > setParamVal (std::string const &paramdescriptor, T const &val)
 
void setParamValUnique (std::string const &paramdescriptor, T const &val)
 
std::vector< std::pair< std::string, T > > getParamVal (std::string const &paramdescriptor) const
 
getParamValUnique (std::string const &paramdescriptor) const
 
std::vector< std::string > setParamString (std::string const &paramdescriptor, std::string const &val)
 
void setParamStringUnique (std::string const &paramdescriptor, std::string const &val)
 
std::vector< std::pair< std::string, std::string > > getParamString (std::string const &paramdescriptor) const
 
std::string getParamStringUnique (std::string const &paramdescriptor) const
 
void freezeParam (std::string const &paramdescriptor)
 
void unFreezeParam (std::string const &paramdescriptor)
 
void freezeAllParams ()
 
void unFreezeAllParams ()
 
std::string descriptor () const
 
void setParamsFromFile (std::string const &filename)
 
std::istream & setParamsFromStream (std::istream &is, std::string const &absfile)
 
void setPath (std::string const &path)
 
std::string absolutePath (std::string const &path="")
 
- Public Member Functions inherited from jevois::ParameterRegistry
virtual ~ParameterRegistry ()
 

Additional Inherited Members

- Protected Member Functions inherited from jevois::Component
virtual void preInit ()
 
virtual void postInit ()
 
virtual void preUninit ()
 
virtual void postUninit ()
 
virtual void preInit ()
 
virtual void postInit ()
 
virtual void preUninit ()
 
virtual void postUninit ()
 
- Protected Member Functions inherited from jevois::ParameterRegistry
void addParameter (ParameterBase *const param)
 
void removeParameter (ParameterBase *const param)
 
void callbackInitCall ()
 

Detailed Description

Abstract base class for an object recognition component.

This base class provides a framework to recognize objects using deep neural networks. The network is implemented using the tiny-dnn library, see https://github.com/tiny-dnn/tiny-dnn

Derived classes implement a particular network by overriding the define() method of the base class. They also must implement the other abstract methods provided here.

To create a new object recognition component, one would usually derive from the ObjectRecognition class template as opposed to deriving directly from ObjectRecognitionBase.

Training

Training is automatically launched if a pre-trained weight file is not found. Thus, typical workflow is, assuming JeVois v1.4 or later:

  • create a new derived class and implement define(), train(), prcess(), etc in there. See ObjectRecognitionMNIST for example.
  • create a directory /jevois/share/tiny-dnn/NAME/ where NAME is the instance name you give to your derived object.
  • copy the training data to that directory (as required by your implementation of train() in your derived class)
  • run jevois-daemon on host, make sure it has write permission to the directory you created. It will fail to load the pre-trained weights and will initiate training.
  • when training is complete, trained weights will be saved as /jevois/share/tiny-dnn/NAME/weights.tnn (for JeVois v1.4 or earlier) or as /jevois/share/tiny-dnn/NAME/weights.tnn.host (for JeVois v1.5 or later).

Starting with JeVois v1.5, tiny-dnn has been updated to a recent version which uses cereal as a back-end to save networks and weights. This yields efficient loading of pre-trained networks but the binary archive format is not portable between Intel x64 hosts and ARM platform. Hence, you should proceed as follows:

  • run the above steps. Two files will be saved: weights.tnn.host (binary) and weights.tnn.json (portable text file).
  • copy weights.tnn.json to microSD in JEVOIS:/share//tiny-dnn/NAME/ where NAME is the instance name you give to your derived object.
  • insert microSD into JeVois camera and launch the machine vision mode that uses your network.
  • ObjectRecognition::load() will fail to load the missing weights.tnn.platform (binary) and will thus revert to loading weights.tnn.json instead. It will then save weights.tnn.platform to microSD. Because this was done by the JeVois camera, weights.tnn.platform will now be in ARM binary format.
  • You can now copy weights.tnn.platform out of your microSD to ~/jevoisbase/share/tiny-dnn/NAME/ so that it will be flashed to microSD next time you make one using jevois-flash-card and you will avoid having to convert again next time JeVois starts.

Definition at line 87 of file ObjectRecognition.H.

Member Typedef Documentation

◆ vec_t

typedef std::vector<tiny_dnn::float_t, tiny_dnn::aligned_allocator<tiny_dnn::float_t, 64> > ObjectRecognitionBase::vec_t

Type used by tiny-dnn for the results:

Definition at line 91 of file ObjectRecognition.H.

Constructor & Destructor Documentation

◆ ObjectRecognitionBase()

ObjectRecognitionBase::ObjectRecognitionBase ( std::string const &  instance)

Constructor.

Definition at line 25 of file ObjectRecognition.C.

◆ ~ObjectRecognitionBase()

ObjectRecognitionBase::~ObjectRecognitionBase ( )
virtual

Virtual destructor for safe inheritance.

Definition at line 30 of file ObjectRecognition.C.

Member Function Documentation

◆ category()

virtual std::string const& ObjectRecognitionBase::category ( size_t  idx) const
pure virtual

Return the name of a given category (0-based index in the vector of results)

Implemented in ObjectRecognitionCIFAR, ObjectRecognitionILAB, and ObjectRecognitionMNIST.

◆ define()

virtual void ObjectRecognitionBase::define ( )
pure virtual

Define the network structure.

Derived classes must implement this function and load a network structure.

Implemented in ObjectRecognitionCIFAR, ObjectRecognitionILAB, and ObjectRecognitionMNIST.

Referenced by ObjectRecognition< tiny_dnn::sequential >::postInit().

◆ insize()

virtual tiny_dnn::index3d<size_t> ObjectRecognitionBase::insize ( ) const
pure virtual

Get the input size for the current network, useful to prepare inputs to process()

Implemented in ObjectRecognition< NetType >, and ObjectRecognition< tiny_dnn::sequential >.

◆ process()

virtual vec_t ObjectRecognitionBase::process ( cv::Mat const &  img,
bool  normalize = true 
)
pure virtual

Process an image, results are confidence for each category.

Implemented in ObjectRecognition< NetType >, and ObjectRecognition< tiny_dnn::sequential >.

◆ train()

virtual void ObjectRecognitionBase::train ( std::string const &  path)
pure virtual

Train the network.

Derived classes must implement this function.

Implemented in ObjectRecognitionCIFAR, ObjectRecognitionILAB, and ObjectRecognitionMNIST.

Referenced by ObjectRecognition< tiny_dnn::sequential >::postInit().


The documentation for this class was generated from the following files: