Welcome new user! You can search existing questions and answers without registering, but please register to post new questions and receive answers. Note that due to large amounts of spam attempts, your first three posts will be manually moderated, so please be patient.
Because of un-manageable amounts of spam despite our use of CAPTCHAs, email authorization, and other tools, we have discontinued this forum (see the 700k+ registered users with validated email addresses at right?). Please email us any questions or post bug reports and feature requests on GitHub at https://github.com/jevois -- The content below remains available for future reference.

General methods to deal with extremely low frame rate

0 votes
Hi,

Maybe a very naive question. I am programming in c++ of a svm based algorithm for handwritten digits recognition. Although I have chosen the default frame rate of 28.5 fps in the configuration file, due to the complexity of the code, the frame rate showing in the guvcview can be extremely slow, down to 2 fps already even without implementing all the processing steps.

Looking at the code, I have used "push_back" function to generate vector several times. Moreover, even if  the training of the svm model is done, loading the model (.xml file) when the training sets are huge can also reduce the frame rate significantly. Video streaming itself, finding contours, choosing valid contours, drawing rectangles and so on are probably also the reasons why the frame rate is so slow.

But in general, are there any methods of optimizing my code? Is it sensible to switch to Python because of the efficiency?

Many thanks!
asked May 11, 2018 in Programmer Questions by yoann747 (250 points)

1 Answer

0 votes
The best would be for us to take a look at your code, maybe if you have it on github or such? Generally, python is slower than C++. All the opencv functions in python are just wrappers around the core opencv which is written in C++. Also, with C++ you can very easily create parallel processing threads.

Here are a few things to consider:

- you should not be loading your XML files on every frame, you can either create a postInit() override in your module and load the files there, or just load them once at the first frame and store the trained classifiers in some data member of your module class. See http://jevois.org/doc/classjevois_1_1Component.html for info about postInit() (Module inherits from Component).

- try the JeVois profiler to see which step is slow: http://jevois.org/doc/classjevois_1_1Profiler.html

For an example of use, search for "itsProfiler" several times in https://github.com/jevois/jevoisbase/blob/master/src/Components/OpticalFlow/FastOpticalFlow.C

The profiler will send messages to serlog. So make sure you have "setpar serlog USB" to see those messages in a serial terminal connected to JeVois.

- try parallel threads. See http://jevois.org/doc/ModuleTutorial.html about running 4 edge detectors in parallel. Using std::async() and lambdas makes it super easy to spawn threads. Just remember to catch the return value of async (a std::future) and to store it into some variable, and then when you need the results of your thread, call get() on that variable. If you do not grab the future returned by async(), its destructor will call get() and will just wait for the thread to complete before proceeding any further. Most of the modules in jevoisbase use threads, so you may want to look at their source for inspiration: http://jevois.org/basedoc/dir_eeb7fcc90d516a232deaaf4de23f9c95.html
answered May 16, 2018 by JeVois (46,580 points)
Hi! Thanks for the detailed reply. I looked into the method that you have mentioned in these days, but since I am not very experienced in C++, I still cannot quite figure it out. The code that I have at the moment is here: https://github.com/yoann747/Integrating-machine-vision-with-JeVois-Camera/blob/master/HelloJeVois.C

You have mentioned to load the xml files at the first stream, which I think could be the solution for this problem. But how exactly can I achieve this? I simply load the xml file outside the process function.  

Besides, I find that in the example of NN ,the way of using postinit() could be applied in my case. Am I right?
http://jevois.org/basedoc/classObjectRecognition.html#details

Could you maybe give a direction so that I can keep working on it? Many thanks!
...