CSI IMX219 camera to work with BeagleY-AI

General Discussion
,
1

Hello,

I am trying to make the Sony IMX219-based camera (RPi Camera Module 2 NoIR) work with BeagleY-AI board.

After adding the overlay reference in the /boot/firmware/extlinux/extlinux.conf file as

label microSD (default)
    kernel /Image
    append console=ttyS2,115200n8 root=/dev/mmcblk1p3 ro rootfstype=ext4 resume=/dev/mmcblk1p2 rootwait net.ifnames=0 quiet
    fdtdir /
    fdt /ti/k3-am67a-beagley-ai.dtb
    fdtoverlays /overlays/k3-am67a-beagley-ai-csi0-imx219.dtbo
    #initrd /initrd.img

rebooting, and running sudo beagle-camera-setup, the camera is detected

$ sudo beagle-camera-setup
IMX219 Camera 0 detected
    device = /dev/video-imx219-cam0
    name = imx219
    format = [fmt:SRGGB8_1X8/1920x1080]
    subdev_id = /dev/v4l-imx219-subdev0
    isp_required = yes

and appears in /dev/

$ ls -l /dev/ | grep video
...
crw-rw---- 1 root video    81,  11 Feb 18 22:43 v4l-subdev2
lrwxrwxrwx 1 root root          11 Feb 18 22:50 video-imx219-cam0 -> /dev/video3
crw-rw---- 1 root video    81,   0 Feb 18 22:43 video0
crw-rw---- 1 root video    81,   1 Feb 18 22:43 video1
crw-rw---- 1 root video    81,   2 Feb 18 22:43 video2
crw-rw---- 1 root video    81,   3 Feb 18 22:43 video3
...

However, when I run a simple test

v4l2-ctl --device /dev/video-imx219-cam0 \
         --set-fmt-video=width=1920,height=1080,pixelformat=YUYV \
         --stream-mmap --stream-count=50 \
         --stream-to=out.yuv

the command hangs indefinitely, and the output file out.yuv is empty.
(The same result is with --device /dev/video3).

Similarly, GStreamer hangs:

gst-launch-1.0 \
  v4l2src device=/dev/video-imx219-cam0 num-buffers=100 ! \
  videoconvert ! \
  x264enc tune=zerolatency bitrate=500 speed-preset=superfast ! \
  mp4mux ! \
  filesink location=output.mp4

…

The dmesg | grep imx219 command produces:

[    6.685482] imx219 5-0010: supply VANA not found, using dummy regulator
[    6.716700] imx219 5-0010: supply VDIG not found, using dummy regulator
[    6.721153] imx219 5-0010: supply VDDL not found, using dummy regulator
[    9.613736] imx219 5-0010: Consider updating driver imx219 to match on endpoints
[   11.768522] Modules linked in: ... imx219 ...
[   11.769904] Modules linked in: ... imx219 ...

I’ve tried editing the k3-am67a-beagley-ai-csi0-imx219-new.dts file (decompiled from and recompiled to the corresponding .dtbo), but no success so far. Any ideas of how to proceed from there?

1 Like
2

Ok, I have actually managed to make it work with these two commands:

media-ctl -V '"imx219 5-0010":0 [fmt:YU12/1920x1080 field:none]'

gst-launch-1.0 -v v4l2src num-buffers=5 device=/dev/video3 io-mode=dmabuf ! \
video/x-bayer, width=1920, height=1080, framerate=30/1, format=rggb ! \
multifilesink location="imx219-image-%d.raw"

The upper command setups the camera ( "imx219 5-0010":0 probably means IMX219 sensor at I2C address 0x10 on bus 5, its first output pad 0), the lower - produces some .raw images.

The format in media-ctl command should apparently be from the outputs of the v4l2-ctl --list-formats-ext. Solution sources: 1, 2.

Additionally, this command works and produces some .yuv video:

v4l2-ctl --device /dev/video3 \
		--set-fmt-video=width=320,height=240,pixelformat=RGGB \
		--stream-mmap \
		--stream-count=50 \
		--stream-to=out.yuv

@lorforlinux Not sure if I address the right person, but should perhaps the “Configuring CSI camera” tutorial be updated?

2 Likes
3

I’m going to make an assumption that you might like to capture to a MP4 file. I recently found that the beagley-ai has a H264 hardware encoder that can be connected to by gstream by using the component v4l2h264enc.

v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl=digital_gain=2048
v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl=analogue_gain=230
v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl=exposure=1750
media-ctl -V '"imx219 5-0010":0[fmt:SRGGB8_1X8/640x480 field:none]'
gst-launch-1.0 v4l2src num-buffers=100 device=/dev/video3 io-mode=dmabuf ! \
video/x-bayer, width=640, height=480, format=rggb,depth=8 ! queue !\
bayer2rgb ! queue ! \
videoconvert ! video/x-raw,format=NV12 ! queue ! \
v4l2h264enc ! queue !\
h264parse ! \
mp4mux ! \
filesink location=imx219.mp4

This manages 30fps but it is low resolution.

Hope that helps

2 Likes
4

I thought it would be useful to add instructions for creating a virtual camera that outputs to standard formats that may be expected in downstream applications (ex. Chromium, OBS, VLC, etc.).

Full code demos can be found here.

Creating a Virtual Camera

In order to run this command, you must install v4l2loopback which enables a kernel
module that can create virtual cameras. This requires:

  1. Installing the Linux Headers for your current kernel. See this article for more information about modifying the kernel.
sudo apt install linux-headers-`uname -r`
  1. Install v4l2loopback-dkms (Dynamic Kernel Module Support). This allows you to add kernel modules without modifying the entire kernel.
sudo apt-get install v4l2loopback-dkms
  1. You may need to reload kernel module dependencies with
sudo depmod -a
  1. Load the newly installed kernel module
sudo modprobe v4l2loopback
  1. Now you can start the virtual camera! This process must be running for the virtual camera to work. You can move the process to the background with your preferred method (ex. screen, ctrl+z, &).
gst-launch-1.0   v4l2src device=/dev/video3 io-mode=dmabuf ! \
    video/x-bayer, width=640, height=480, framerate=30/1, format=rggb,depth=8 ! \
    bayer2rgb ! videoconvert ! video/x-raw, format=NV12 ! \
    v4l2sink device=/dev/video9

This will create a virtual camera on /dev/video9 or index 9 that can be recognized
by downstream applications.

1 Like
5

Amm… So does the ISP working with the IMX219 on BeagleY-AI? I mean can you control white balance, exposure, etc.? Or is it still just a raw sensor data?

1 Like