Still having problems with custom weights

Question

Thanks for the great support. Your last answer was very helpful. I found that serout is not a valid command in the script, but the putting the YOLO config parameters in the script got it started.

I think I'm crashing the device. With INF set to USB, I get the following when I load Yolo:

INF Engine::setFormatInternal: OUT: YUYV 1280x480 @ 15fps CAM: YUYV 640x480 @ 15fps MOD: JeVois:DarknetYOLO
INF Camera::setFormat: Camera set video format to 640x480 YUYV
INF Gadget::setFormat: USB Gadget set video format to 1280x480 YUYV
INF Engine::setFormatInternal: Instantiating dynamic loader for /jevois/modules/JeVois/DarknetYOLO/DarknetYOLO.so
OK
INF Engine::setFormatInternal: Module [DarknetYOLO] loaded, initialized, and ready.
INF Yolo::operator(): Using data config from /jevois/share/darknet/yolo/cfg/pdq_obj.data
INF Yolo::operator(): Using cfg from /jevois/share/darknet/yolo/cfg/pdq.cfg
INF Yolo::operator(): Using weights from /jevois/share/darknet/yolo/weights/pdq_3100.weights
INF Yolo::operator(): Using names from /jevois/share/darknet/yolo/data/pdq_obj.names
INF Yolo::operator(): Getting labels...
INF Yolo::operator(): Parsing network and loading weights...

At that point, the amber light turns off in the camera and it no longer responsive to any commands.

Any suggestions? 

I thought it might be a problem with the network being to large so I retrained using tiny yolo as a basis. The weights file is now about the same size as the others, yet it fails in the same way.

One more dumb question (for right now). What version of YOLO is this? What version of Yolo-tiny is it? There are multiple versions on github here: https://github.com/pjreddie/darknet/tree/master/cfg

My next step is to grab the working tiny-voc config file and use it as the basis for training my data.

Answer 1

Yes, you may be running out of memory. You may want to try with a lower resolution first. Have you tried to set the netin parameter to something small, like 160x120?

Do you have a USB to serial cable that you could connect to the hardware serial port of JeVois? It is useful to debug hard crashes. It would tell you whether you are running out of memory or just getting some error in the YOLO code.

Otherwise, can you send us your weights somehow (e.g., google drive, share with jevoisinc@gmail.com)? We can then try to load them and use the USB to serial cable and run gdb on the camera to investigate what is happening.

See here for some more info on debugging: http://jevois.org/doc/Debugging.html

Comments

Thank you. I've tried smaller weight files with the same result. I will try lower res. and if I can't solve it I'll send you my weight file.

Can you confirm that the cam has Yolo V3 installed and not an earlier version?

---

oh, sorry, I forgot, it is YOLO v2, which could also be the source of your problems if you are using weights for V3. We will update to V3, it's on our todo list, but we have a few other urgent things to finish first...

---

Ok. Retrained with tiny YoloV2. It's still failing but it's getting a little farther. Last debug message now is:
INF Engine::setFormatInternal: Module [DarknetYOLO] loaded, initialized, and ready.
Terminating Log activity
The the amber light goes out and the device is unreachable. I don't have a usb-serial cable here, but I'll get one so I can do more debugging.

---

Ok. Thanks to an email from laurent, I have a solution. Using tiny Yolov2, I needed to have /n line terminations in params.cfg not windows style cr/lf.
<<update>> Maybe not. I can no longer reproduce the problem. It's not as simple as just different line endings. Perhaps there was some other garbage in my params file.