Welcome new user! You can search existing questions and answers without registering, but please register to post new questions and receive answers. Note that due to large amounts of spam attempts, your first three posts will be manually moderated, so please be patient.
We have moved to a new forum at http://jevois.usc.edu, please check it out. The forum at jevois.org/qa will not allow new user registrations but is maintained alive for its useful past questions and answers.
Welcome to JeVois Tech Zone, where you can ask questions and receive answers from other members of the community.

Can the image be processed in raw image format?

0 votes
We would like to process the image in YCbCr format, as far as I understand this is the digital version of YUYV which should come directly from the sensor. Is it possible to process the image directly in that color format or is the conversion -->BGR -->YCbCr required?
asked Aug 13, 2018 in Programmer Questions by phildue (420 points)

1 Answer

0 votes
 
Best answer
Yes, we do that quite a lot, though mostly for drawing as opposed to deeper processing.

See for example

https://github.com/jevois/jevoisbase/blob/master/src/Modules/TensorFlowEasy/TensorFlowEasy.C

Getting the central ROI to be sent to the neural net, and doing all the drawings, is done directly in YUYV.

However, in that example, we do convert the contents of the ROI to RGB because the deep network was trained using RGB.

In principle, you could re-train the deep net to work direclty with YUYV (or even raw Bayer) inputs, but we have not tried it so we don't know how well it will work. Also note that our tensorflow module currently expects either rgb or gray, so it would have to be extended to also accept YUYV inputs.

The following functions operate on raw YUYV; look in the various jevoisbase modules for examples of how to use them:

http://jevois.org/doc/RawImageOps_8H.html
answered Aug 20, 2018 by JeVois (46,580 points)
selected Sep 6, 2018 by phildue
Thanks for the answer. I tried it this way but it seems that the cv::Mat obtained from jevois::rawimage::cvImage only has two channels, so the data type is 8UC2.

Wouldn't it need at least 3 channels?
How can I continue processing this format?

Edit: Understood now the encoding and checked the source code and found the appropriate functions:)

If someone else needs this: the 1rst channel is multiplexed so it contains odd columns U and even columns V, while Y is the 0th channel
...