Read Only File System after sendSerial?

Question

I've had this happen twice now, on two different JeVois cameras, two different SD cards.  One of the SD cards was a year old, so had an old Jevois image.  The other was flashed brand new today.

I have a script running.  It's a fairly straightforward First Robotics type script.
Start the script.
Open up a new text file to save some data about the coming run.
Start acquiring and processing images.  Find targets and draw the target outline on the screen.  Save target locations in the text file mentioned above.
Send processed frame via outframe to the host.

So far so good.  It runs.  I see exactly what I expect to see in Inventor, or whatever happens to be viewing the output stream.

Now......add one line:
jevois.sendSerial("A 1.23 ")

(That is in fact the exact line I'm sending.  Of course in the future I will send target coordinates instead of "1.23", but I'm testing serial communications now.)

Restart jevois, and in inventor, it now tells me it can't open the text file because it's a read only file system.  Throw exception and stop.

Well now that's odd.
Try to take out the sendSerial line by editing in Inventor.  It can't save the code.  The file system is read only.

 The first time it happened I thought it must be coincidence and somehow I managed to corrupt the SD card with something unrelated, so I dug out another camera, flashed a brand new SD card, and exactly the same thing happened.  This time I opened the file system in Windows, went into the module code and manually edited it out, ejected the disk, and brought it up in inventor again.  Same error message.  Looking at the code, the sendSerial line is gone, but I still can't run the program or save any changes.  The file system appears to be read only, at least to the jeVois software.  I can open it and write to it in Windows (via enabling access to the file system, from the Inventor button.)

The exact error message is OSError: [errno 30]  Read-only file system: 'targetData.txt'

(targetData.txt is the name of the file that I open up at the beginning to save found target data.)
Greatly abbreviated code below:
import libjevois as jevois
import cv2
import numpy as np
import random
import math

class RoboCon_320_240_30:
def __init__(self):
...Do a bunch of initialization

   self.datafile=open("targetData.txt","w+")

def substance(self,inimg):
...Do the major processing.  Called from either process or processNoUSB

def process(self, inframe, outframe):
   inimg=inframe.getCvBGR()
   ….Do a bunch of stuff to it
   ........call substance()
   jevois.sendSerial("A 1.23 ")
   outframe.sendCv(processedimage)

Everything is fine, until I insert that sendSerial line.  Then it stops working and the file system is read only.

Any ideas?

Answer 1

Can you use an absolute path when you open the file, like "/jevois/targetData.txt"?

It might be that for some strange reason you do not always end up being in the module's directory as you should be (maybe something funny going on during instantiation of a python module, the constructor may be called before we change directory to the module's directory). The /jevois partition should be writable.

Or, in the inventor console, try

shell dmesg

and look for messages that would signal some disk error and re-mounting read-only. Or try

shell touch /jevois/hello

and then

shell ls /jevois

and see if you get errors and whether the file was created.

If that does not help, we have had strange situations also (but hard to reproduce) when trying to save python code to SD repeatedly and across multiple days. I think it might have to do with date and pycache. And maybe a broken pycache triggers the remount to read-only. So I would recommend the following: after JeVois boots up, fire up the inventor and go to the System tab and set the date/time on the camera to that of your host. Then you should have no problem editing, etc. My guess about this is that since by default jevois starts on Jan 1, 1970 (since there is no battery for the clock), depending on when you edit and try to save, you may end up with saving the source python file earlier than you did last time you booted up JeVois, and so now you end up with the compiled python code in pycache being in the future compared to the time at which you save your source python file. That may upset python.

Comments

I didn't try all of the above.  Before I tried any of those, I decided to separate the sendSerial command from the command to send the image.  I know that "shouldn't" cause a problem, but it was a hunch.  I stuck an OpenCV command in between them and the problem went away.

It may rear its ugly head again in the future, but hopefully if it does, I will have time to deal with it.  Right now, I've moved on to my next problem, which is how to communicate via serial over usb.

---

Sigh.

After writing the above, I went back to do more testing and......it came back.  Read only file system.


Here's a long shot, but......I have a 64G micro SD stuck in these cameras.  Good idea?  Been tested with that size?  I'm only using them because they got dirt cheap and that's what I had available.  I certainly don't need 64 Gig of storage on that camera.

---

Tried with an 8 gig card.  It lasted a while, but eventually went read only.  Here's some messages:

shell dmesg
[    0.000000] Booting Linux on physical CPU 0
[    0.000000] Linux version 3.4.39 (itti@iLab0) (gcc version 4.7.4 (Ubuntu/Linaro 4.7.4-3ubuntu12) ) #10 SMP Mon Sep 9 11:32:35 PDT 2019
[    0.000000] CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] Machine: sun8i
[    0.000000] Memory policy: ECC disabled, Data cache writealloc
[    0.000000] On node 0 totalpages: 65536
[    0.000000] free_area_init_node: node 0, pgdat c046a3c0, node_mem_map c04d2000
[    0.000000]   Normal zone: 512 pages used for memmap
[    0.000000]   Normal zone: 0 pages reserved
[    0.000000]   Normal zone: 65024 pages, LIFO batch:15
[    0.000000] script_init enter!
…snip due to message size limit....
[    1.353928] [VFE]Check sensor!
[    1.390071] [VFE]mclk on
[    1.510736] [OV9650] Detected Omnivision OV965x camera: ov9652
[    1.554933] [VFE]jevois: sensor detect ok
[    1.554944] [VFE]mclk off
[    1.592213] [VFE]Sub device register "ov9650" is OK!
[    1.592528] [VFE]V4L2 device registered as video0
[    1.592562] [VFE]probe_work_handle end!
[    1.592601] [VFE]vfe_probe success!
[    1.592732] [VFE]vfe_init end
[    1.604555] UMP: UMP device driver 0000 loaded
[    1.670276] [DISP]disp_module_init
[    1.691596] [fb_map_boot_logo]w=0,h=0,addr=0x0
[    1.691616] vmalloc: allocation failure: 0 bytes
[    1.691628] insmod: page allocation failure: order:0, mode:0xd2
[    1.691640] [DISP] Fb_map_boot_logo,line:440:    Fb_map_boot_logo err: Fb_map_kernel failed!
[    1.694949] [DISP]disp_module_init finish
[    1.719278] =========================================================
[    1.719296]        Mali GPU Information         
[    1.719357] voltage             : 1140 mV
[    1.719367] initial frequency   : 408000000 MHz
[    1.719377] frame buffer address: 0x4e802000 - 0x56802000
[    1.719386] frame buffer size   : 128 MB
[    1.719394] =========================================================
[    1.721549] Mali: Mali device driver loaded
[    3.399281] g_jevoisa33 gadget: Mass Storage Function, version: 2009/09/11
[    3.399301] g_jevoisa33 gadget: Number of LUNs=1
[    3.399350]  lun0: LUN: removable file: (no medium)
[    3.399380] PARSEMODES: found 1 modes and 1 frames
[    3.399430] g_jevoisa33 gadget: uvc_function_bind
[    3.399782] g_jevoisa33 gadget: JeVois-A33 Smart Camera Gadget
[    3.399804] g_jevoisa33 gadget: g_jevoisa33 ready
[    5.561379] [VFE]vfe_open
[    5.561397] [VFE]..........................vfe clk open!.......................
[    5.561421] [VFE]vfe_open ok
[    5.561489] [VFE_ERR]input index(1) > dev->dev_qty(1)-1 invalid!
[    5.561944] [VFE]Set vfe core clk = 108000000, after Set vfe core clk = 99000000
[    5.600031] [VFE]mclk on
[    5.720705] [OV9650] Detected Omnivision OV965x camera: ov9652
[    5.800893] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[    5.800909] FAT-fs (mmcblk0p3): Filesystem has been set read-only
[    5.800974] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[    5.800989] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[    5.801004] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[    5.801018] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[    5.801032] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[    5.801046] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[    5.802870] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[    5.802888] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[    5.802903] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[    6.620115] g_jevoisa33 gadget: high-speed config #1: Video
[    6.674589] WRN:L859(drivers/usb/sunxi_usb/udc/sunxi_udc.c):handle_ep0: ep0 setup end req=cd93e7c0
[   17.476031] WRN:L859(drivers/usb/sunxi_usb/udc/sunxi_udc.c):handle_ep0: ep0 setup end req=cd93e7c0
[   17.804505] fat_get_cluster: 21 callbacks suppressed
[   17.804523] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[   17.804540] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[   17.804555] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[   17.804569] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[   17.804583] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[   17.804597] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[   17.804625] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[   17.804637] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[   17.804652] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
[   17.804666] FAT-fs (mmcblk0p3): error, fat_get_cluster: invalid cluster chain (i_pos 10581393)
OK




Don't know if that tells you anything.

In the meantime, before it went read only I could see messages on the inventor console when I plugged in the camera, but never saw anything else.  i.e. over on my Roborio, it never received a character.

Tomorrow I'll try troubleshooting.  Maybe a windows terminal program.  Maybe a different power source.  Tonight, I'm depressed.  It's late.  I have to work in the morning, and my robot still can't see.

Oh, and when I tried the touch file:


shell touch /jevois/hello
touch: /jevois/hello: Read-only file system
OK

---

darn, so you are indeed getting errors on the card, which is why the OS of JeVois turns it to read only. Is that one of our cards? We flash them with full sector-by-sector copy + compare so we reject those that fail before we ship (has not been happening much lately though).

[    5.800909] FAT-fs (mmcblk0p3): Filesystem has been set read-only

Do you have another microSD that you could try? We use SanDisk Extreme Pro 32GB here for development, can be flashed in 1 minute flat (using jevois-flash-card from a compiled source tree as opposed to copying a distro image, which may take longer).

---

It's not one of your cards.  More accurately, none of the several cards I have used have been your cards.  (At least, I don't think.  I have several of your cameras, but I think I only bought one "kit".)

I've been using your cameras for a little over a year, and I've never had this problem.  It's even running code that was run at a robotics competition in July, where it rode the robot and did all of the same fun things it's doing now, except I added one additional line of sendSerial.  I got about 20,000 pictures off of it that day.  It saved the targetdata file.  It saved the received images.  It communicated with the Roborio (although the Roborio wasn't listening at the time.  We didn't actually have the vision integrated with our robot motion.  We were just testing the algorithms.)

I'm using the same copy of Balena etcher to flash the SD cards.  One of the cards I used was a 64 Gig.  Two others were 8 gig.  The only difference I know of between last summer and now is that last summer I was running earlier versions of software on both the Jevois and the Roborio.  I was using version 12, I think, last summer.  It would tell me there was a new version available, but I ignored it.  A few weeks ago, I downloaded Version 1.14 from your site.

I have also upgraded the system software of the Roborio to the 2020 edition, and of course running different robot code.


Tonight's new wrinkle is that I decided to take out the line where it sent the processed frame out.  I didn't go all the way to headless mode, but I wasn't actually streaming the video.  Just sending serial data out.  The card ended up read only.

---

Maybe your SD connector has gone bad somehow. Please email us at jevoisinc@gmail.com and we will ship a new camera to you to try out.

---

Thanks.  But the problem has occurred on three different Jevois cameras.

I realized a couple of things yesterday.  First, I've left it running for a very long time on Inventor, without the problem ever developing.  On the Roborio, it dies.

Second, I'm starting my SerialPort object on the Roborio in the RobotInit() function.  I don't know how long it takes between the time the Roborio is booted, and the time that function initializes, but I wonder if it is possible that the camera isn't ready at the time it starts, and that something in the camera's serial driver is failing and corrupting the file system.

For my next trick, I will delay the start of the SerialPort on the rio to give enough time to be sure the Jevois has fully booted and is ready to send data.