General methods to deal with extremely low frame rate

Question

Hi,

Maybe a very naive question. I am programming in c++ of a svm based algorithm for handwritten digits recognition. Although I have chosen the default frame rate of 28.5 fps in the configuration file, due to the complexity of the code, the frame rate showing in the guvcview can be extremely slow, down to 2 fps already even without implementing all the processing steps.

Looking at the code, I have used "push_back" function to generate vector several times. Moreover, even if  the training of the svm model is done, loading the model (.xml file) when the training sets are huge can also reduce the frame rate significantly. Video streaming itself, finding contours, choosing valid contours, drawing rectangles and so on are probably also the reasons why the frame rate is so slow.

But in general, are there any methods of optimizing my code? Is it sensible to switch to Python because of the efficiency?

Many thanks!

Answer 1

The best would be for us to take a look at your code, maybe if you have it on github or such? Generally, python is slower than C++. All the opencv functions in python are just wrappers around the core opencv which is written in C++. Also, with C++ you can very easily create parallel processing threads.

Here are a few things to consider:

- you should not be loading your XML files on every frame, you can either create a postInit() override in your module and load the files there, or just load them once at the first frame and store the trained classifiers in some data member of your module class. See http://jevois.org/doc/classjevois_1_1Component.html for info about postInit() (Module inherits from Component).

- try the JeVois profiler to see which step is slow: http://jevois.org/doc/classjevois_1_1Profiler.html

For an example of use, search for "itsProfiler" several times in https://github.com/jevois/jevoisbase/blob/master/src/Components/OpticalFlow/FastOpticalFlow.C

The profiler will send messages to serlog. So make sure you have "setpar serlog USB" to see those messages in a serial terminal connected to JeVois.

- try parallel threads. See http://jevois.org/doc/ModuleTutorial.html about running 4 edge detectors in parallel. Using std::async() and lambdas makes it super easy to spawn threads. Just remember to catch the return value of async (a std::future) and to store it into some variable, and then when you need the results of your thread, call get() on that variable. If you do not grab the future returned by async(), its destructor will call get() and will just wait for the thread to complete before proceeding any further. Most of the modules in jevoisbase use threads, so you may want to look at their source for inspiration: http://jevois.org/basedoc/dir_eeb7fcc90d516a232deaaf4de23f9c95.html

Comments

Hi! Thanks for the detailed reply. I looked into the method that you have mentioned in these days, but since I am not very experienced in C++, I still cannot quite figure it out. The code that I have at the moment is here: https://github.com/yoann747/Integrating-machine-vision-with-JeVois-Camera/blob/master/HelloJeVois.C

You have mentioned to load the xml files at the first stream, which I think could be the solution for this problem. But how exactly can I achieve this? I simply load the xml file outside the process function.  

Besides, I find that in the example of NN ,the way of using postinit() could be applied in my case. Am I right?
http://jevois.org/basedoc/classObjectRecognition.html#details

Could you maybe give a direction so that I can keep working on it? Many thanks!