IMU triggered shutter ar0135

Question

Hello,

Looking at the new hardware features available and more particulary :

• OnSemi AR0135
• IMU TDK/InvenSense ICM-20948.

I am curious to know if we can get the global shutter being trigged by the imu in DMP mode qith a prescaler to have various trigger rate from 10 to 40 Hz.

That would make a basic setup to implement  VIO : Visual Inertial Odometry

Here is my prototype , that I would be interested to ''scale down"  to a Jevois configuration:

https://discuss.ardupilot.org/t/experiment-with-visual-odometry-rovio/40120

Regards

Answer 1

Super nice project, thanks for sharing, amazing!

The ICM-20948 natively provides the opposite capability: there is an FSYNC input that you can connect to the VSYNC output of the camera (I am guessing the MPU-9250 has it too?). We are currently exploring this for a future version of the sensor. In principle it should work, but it is very unclear from the docs or the sample code from TDK exactly how (they do not implement anything related to FSYNC it seems). Presumably, the DMP will issue a message or set a flag in the header1 or header2 of existing messages each time an FSYNC pulse is received from the camera.

Would that be usable? We just got the new boards back with that VSYNC-FSYNC connection setup and will be testing it soon and writing a driver for it.

Comments

I double checked and we can use the ICM interrupts the same way it is used on the MPU-9250. It may sound a little nonsense but the interrupt is an OUT signal, that is interfaced to the Arduino to trig  the programmable prescaler in order to fire the camera Sync Shutter.

The major difference reside in the signal level
MPU-9250: 1.71V to 3.6V
ICM-20948: 1.71V to 1.95V

ref.  
http://www.invensense.com/wp-content/uploads/2018/10/AN-000146-v2.0-TDK_Migration_MPU_9250toICM-20948.pdf

https://maker.pro/arduino/tutorial/how-to-interface-arduino-and-the-mpu-6050-sensor

---

Sounds good, though this is less desirable from our perspective (we need to get both the interrupt pin and the camera sensor's VSYNC trigger out to the world somehow, which poses problems with ESD safety and with FCC/CE/etc).

So what about turning your logic around as I suggested, could this work? That is, the sensor sets the frame rate. But the IMU knows when the next frame is about to be captured (when we are in vertical blanking) and sends a message about this over I2C.

---

That would work ok in a one-to-one IMU camera Frame scenario.
Most of the use cases for VIO are based on high speed IMU preintegration with lower rate Images that are fused within the EKF. So getting the Sync from the camera could induce drifting within the IMU sampling.

https://www.semanticscholar.org/paper/IMU-Preintegration-on-Manifold-for-Efficient-Forster-Carlone/25e26d4db49e280669e26cd05305d27f35a0791a/figure/3


https://pdfs.semanticscholar.org/a4e4/87dbfd68f1969d03ff6b1321ffe9ac782639.pdf?_ga=2.94582746.755165965.1555440979-797795228.1555440979

---

I see, thanks, very interesting. Indeed having the IMU in control ensures a constant number of samples in each preintegration period. Also if you have more than one camera it would be more desirable to sync all of them up to a single VSYNC, which could be produced by the IMU plus a prescaler. Let me think about it but this would require yet another hardware revision, which takes a lot of time. In passing, we should look at how well SVO runs on jevois, that's a great idea for a new module.