JeVoisBase  1.0
JeVois Smart Embedded Machine Vision Toolkit Base Modules
ObjectRecognition.H
Go to the documentation of this file.
1 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
2 //
3 // JeVois Smart Embedded Machine Vision Toolkit - Copyright (C) 2016 by Laurent Itti, the University of Southern
4 // California (USC), and iLab at USC. See http://iLab.usc.edu and http://jevois.org for information about this project.
5 //
6 // This file is part of the JeVois Smart Embedded Machine Vision Toolkit. This program is free software; you can
7 // redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software
8 // Foundation, version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
9 // without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
10 // License for more details. You should have received a copy of the GNU General Public License along with this program;
11 // if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
12 //
13 // Contact information: Laurent Itti - 3641 Watt Way, HNB-07A - Los Angeles, CA 90089-2520 - USA.
14 // Tel: +1 213 740 3527 - itti@pollux.usc.edu - http://iLab.usc.edu - http://jevois.org
15 // ///////////////////////////////////////////////////////////////////////////////////////////////////////////////////
16 /*! \file */
17 
18 #pragma once
19 
21 #include <stdarg.h> // needed by tiny_dnn
22 
23 // Defines used to optimize tiny-dnn:
24 #define CNN_USE_TBB
25 #undef CNN_USE_DOUBLE
26 
27 #include <tiny-dnn/tiny_dnn/config.h> // for float_t, etc. this does not include much code
28 #include <tiny-dnn/tiny_dnn/util/aligned_allocator.h> // for aligned_allocator
29 #include <tiny-dnn/tiny_dnn/util/util.h> // for index3d
30 #include <opencv2/core/core.hpp>
31 
32 namespace tiny_dnn { template <typename NetType> class network; }
33 
34 //! Abstract base class for an object recognition component
35 /*! This base class provides a framework to recognize objects using deep neural networks. The network is implemented
36  using the tiny-dnn library, see https://github.com/tiny-dnn/tiny-dnn
37 
38  Derived classes implement a particular network by overriding the define() method of the base class. They also must
39  implement the other abstract methods provided here.
40 
41  To create a new object recognition component, one would usually derive from the ObjectRecognition class template as
42  opposed to deriving directly from ObjectRecognitionBase.
43 
44  \ingroup components */
46 {
47  public:
48  //! Type used by tiny-dnn for the results:
49  typedef std::vector<tiny_dnn::float_t, tiny_dnn::aligned_allocator<tiny_dnn::float_t, 64> > vec_t;
50 
51  //! Constructor
52  ObjectRecognitionBase(std::string const & instance);
53 
54  //! Virtual destructor for safe inheritance
55  virtual ~ObjectRecognitionBase();
56 
57  //! Define the network structure
58  /*! Derived classes must implement this function and load a network structure. */
59  virtual void define() = 0;
60 
61  //! Get the input size for the current network, useful to prepare inputs to process()
62  virtual tiny_dnn::index3d<tiny_dnn::serial_size_t> insize() const = 0;
63 
64  //! Train the network
65  /*! Derived classes must implement this function. */
66  virtual void train(std::string const & path) = 0;
67 
68  //! Process an image, results are confidence for each category
69  virtual vec_t process(cv::Mat const & img, bool normalize = true) = 0;
70 
71  //! Return the name of a given category (0-based index in the vector of results)
72  virtual std::string const & category(size_t idx) const = 0;
73 };
74 
75 //! Wrapper around a neural network implemented by with the tiny-dnn framework by Taiga Nomi
76 /*! Because tiny-dnn is an all-include package, we use the pimpl idiom here to avoid including all the tiny-dnn
77  sources in the header file, and instead only include and compile them once in our ObjectRecognition.C file.
78 
79  \ingroup components */
80 template <typename NetType>
82 {
83  public:
84  //! Constructor allocates the (empty) network
85  ObjectRecognition(std::string const & instance);
86 
87  //! Destructor
88  virtual ~ObjectRecognition();
89 
90  //! Get the input size for the current network, useful to prepare inputs to process()
91  virtual tiny_dnn::index3d<tiny_dnn::serial_size_t> insize() const override;
92 
93  //! Process an image, results are confidence for each category
94  vec_t process(cv::Mat const & img, bool normalize = true) override;
95 
96  protected:
97  //! Initialize the network, required before one starts using it
98  /*! First, we will call define(). Then, we will look in path for weights.tnn, and if not found, we will call
99  train() to train the network using data in that path, and then we will save weights.tnn. Derived classes may
100  implement a constructor that takes path and then calls init(path) after the base class has been constructed
101  (e.g., in the body of the derived class constructor). */
102  virtual void postInit() override;
103 
104  tiny_dnn::network<NetType> * net; // pointer here to avoid #include'ing tiny_dnn.h
105 };
ObjectRecognitionBase(std::string const &instance)
Constructor.
virtual std::string const & category(size_t idx) const =0
Return the name of a given category (0-based index in the vector of results)
virtual vec_t process(cv::Mat const &img, bool normalize=true)=0
Process an image, results are confidence for each category.
virtual void postInit() override
Initialize the network, required before one starts using it.
vec_t process(cv::Mat const &img, bool normalize=true) override
Process an image, results are confidence for each category.
virtual ~ObjectRecognitionBase()
Virtual destructor for safe inheritance.
virtual void define()=0
Define the network structure.
virtual tiny_dnn::index3d< tiny_dnn::serial_size_t > insize() const override
Get the input size for the current network, useful to prepare inputs to process() ...
virtual tiny_dnn::index3d< tiny_dnn::serial_size_t > insize() const =0
Get the input size for the current network, useful to prepare inputs to process() ...
std::vector< tiny_dnn::float_t, tiny_dnn::aligned_allocator< tiny_dnn::float_t, 64 > > vec_t
Type used by tiny-dnn for the results:
virtual ~ObjectRecognition()
Destructor.
ObjectRecognition(std::string const &instance)
Constructor allocates the (empty) network.
Abstract base class for an object recognition component.
Wrapper around a neural network implemented by with the tiny-dnn framework by Taiga Nomi...
virtual void train(std::string const &path)=0
Train the network.
tiny_dnn::network< NetType > * net