Yes the demos do work with the Edge AI demo image. Images + Video classification is working.
I additionally attached the USB serial adapter and got a serial console. Then I killed the /usr/bin/edgeai-gui-app -platform linuxfb process and restarted it from the console to get some output on stdout/stderr. Then I
- first started a Video classification from “Custom” menu → Video and face recognition was visible
- closed it by clicking on “Custom”
- started a Camera classification with “IMX219 Camera 0” (same model) → nothing showed up, the “Custom” window closed
- closed it by clicking on “Custom”.
This is what I got on the serial line:
root@j722s-evm:/opt/edgeai-gst-apps# /usr/bin/edgeai-gui-app -platform linuxfb
Detected locale "C" with character encoding "ANSI_X3.4-1968", which is not UTF-8.
Qt depends on a UTF-8 locale, and has switched to "C.UTF-8" instead.
If this causes problems, reconfigure your locale. See the locale(1) manual
for more information.
Input Type = Video;
Input File = /opt/edgeai-test-data/videos/oob-gui-video8.h264;
Model = /opt/model_zoo/ONR-OD-8420-yolox-s-lite-mmdet-widerface-640x640
[ 67.468722] vdec 30210000.video-codec: wave5_vpu_firmware_command_queue_error_check: result not ready: 0x800
[ 67.478601] vdec 30210000.video-codec: wave5_vpu_dec_finish_decode: could not get output info.
[ 67.478660] vdec 30210000.video-codec: wave5_vpu_firmware_command_queue_error_check: result not ready: 0x800
[ 67.497042] vdec 30210000.video-codec: wave5_vpu_firmware_command_queue_error_check: result not ready: 0x800
[ 67.506888] vdec 30210000.video-codec: wave5_vpu_dec_finish_decode: could not get output info.
Custom Pipeline:
multifilesrc location=/opt/edgeai-test-data/videos/oob-gui-video8.h264 loop=true stop-index=-1 ! h264parse ! v4l2h264dec capture-io-mode=5 ! tiovxmemalloc pool-size=12 ! video/x-raw, format=NV12 ! tiovxmultiscaler name=split_01 src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1280 src_0::roi-height=768 target=0 split_01. ! queue ! video/x-raw, width=640, height=640 ! tiovxdlpreproc model=/opt/model_zoo/ONR-OD-8420-yolox-s-lite-mmdet-widerface-640x640 out-pool-size=4 ! application/x-tensor-tiovx ! tidlinferer target=1 model=/opt/model_zoo/ONR-OD-8420-yolox-s-lite-mmdet-widerface-640x640 ! post_0.tensor split_01. ! queue ! video/x-raw, width=1280, height=768 ! post_0.sink tidlpostproc name=post_0 model=/opt/model_zoo/ONR-OD-8420-yolox-s-lite-mmdet-widerface-640x640 alpha=0.200000 viz-threshold=0.500000 top-N=5 display-model=true ! queue ! mosaic_0. tiovxmosaic name=mosaic_0 target=1 src::pool-size=4 sink_0::startx="<320>" sink_0::starty="<131>" sink_0::widths="<1280>" sink_0::heights="<768>" ! video/x-raw,format=NV12, width=1920, height=1080 ! queue ! tiperfoverlay main-title=null title="oob-gui-video8.h264->ONR-OD-8420-yolox-s-lite-mmdet-widerface-640x640" overlay-type=graph ! tiovxmultiscaler ! video/x-raw, width=1198,height=668 ! queue max-size-buffers=1 ! kmssink driver-name=tidss name="sink" render-rectangle="<222,133,1198,668>" sync=false
APP: Init ... !!!
1578.326309 s: MEM: Init ... !!!
1578.326377 s: MEM: Initialized DMA HEAP (fd=32) !!!
1578.326542 s: MEM: Init ... Done !!!
1578.326557 s: IPC: Init ... !!!
1578.384911 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
1578.397477 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
1578.397699 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
1578.397718 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
1578.397729 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
1578.398469 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0
1578.398644 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1
1578.398781 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2
1578.398902 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3
1578.398920 s: VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!!
1578.398933 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
libtidl_onnxrt_EP loaded 0x28f3fab0
Final number of subgraphs created are : 1, - Offloaded Nodes - 257, Total Nodes - 257
(edgeai-gui-app:1472): GLib-GObject-CRITICAL **: 15:38:34.157: g_object_set_is_valid_property: object class 'GstKMSSink' has no property named 'widget'
Starting video
Stopping video
[ 87.990517] vdec 30210000.video-codec: wave5_vpu_firmware_command_queue_error_check: result not ready: 0x800
Removing gst_pipeline
1598.334411 s: VX_ZONE_WARNING: [vxReleaseContext:1439] Found a reference 0xffff87ea20c8 of type 0000080f at external count 1, internal count 0, releasing it
1598.334512 s: VX_ZONE_WARNING: [vxReleaseContext:1441] Releasing reference (name=image_122) now as a part of garbage collection
1598.334539 s: VX_ZONE_WARNING: [vxReleaseContext:1439] Found a reference 0xffff87f206b0 of type 00000813 at external count 1, internal count 0, releasing it
1598.334553 s: VX_ZONE_WARNING: [vxReleaseContext:1441] Releasing reference (name=object_array_123) now as a part of garbage collection
1598.334831 s: VX_ZONE_WARNING: [vxReleaseContext:1439] Found a reference 0xffff87f20868 of type 00000813 at external count 1, internal count 0, releasing it
1598.334848 s: VX_ZONE_WARNING: [vxReleaseContext:1441] Releasing reference (name=object_array_125) now as a part of garbage collection
1598.335043 s: VX_ZONE_WARNING: [vxReleaseContext:1439] Found a reference 0xffff87f20a20 of type 00000813 at external count 1, internal count 0, releasing it
1598.335057 s: VX_ZONE_WARNING: [vxReleaseContext:1441] Releasing reference (name=object_array_127) now as a part of garbage collection
1598.335250 s: VX_ZONE_WARNING: [vxReleaseContext:1439] Found a reference 0xffff87f20bd8 of type 00000813 at external count 1, internal count 0, releasing it
1598.335266 s: VX_ZONE_WARNING: [vxReleaseContext:1441] Releasing reference (name=object_array_129) now as a part of garbage collection
1598.335492 s: VX_ZONE_WARNING: [vxReleaseContext:1439] Found a reference 0xffff87f20d90 of type 00000813 at external count 1, internal count 0, releasing it
1598.335507 s: VX_ZONE_WARNING: [vxReleaseContext:1441] Releasing reference (name=object_array_131) now as a part of garbage collection
1598.335706 s: VX_ZONE_WARNING: [vxReleaseContext:1439] Found a reference 0xffff87f20f48 of type 00000813 at external count 1, internal count 0, releasing it
1598.335721 s: VX_ZONE_WARNING: [vxReleaseContext:1441] Releasing reference (name=object_array_133) now as a part of garbage collection
1598.335906 s: VX_ZONE_WARNING: [vxReleaseContext:1439] Found a reference 0xffff87f21100 of type 00000813 at external count 1, internal count 0, releasing it
1598.335921 s: VX_ZONE_WARNING: [vxReleaseContext:1441] Releasing reference (name=object_array_135) now as a part of garbage collection
1598.336108 s: VX_ZONE_WARNING: [vxReleaseContext:1439] Found a reference 0xffff87f212b8 of type 00000813 at external count 1, internal count 0, releasing it
1598.336123 s: VX_ZONE_WARNING: [vxReleaseContext:1441] Releasing reference (name=object_array_137) now as a part of garbage collection
1598.336313 s: VX_ZONE_WARNING: [vxReleaseContext:1439] Found a reference 0xffff87f21470 of type 00000813 at external count 1, internal count 0, releasing it
1598.336328 s: VX_ZONE_WARNING: [vxReleaseContext:1441] Releasing reference (name=object_array_139) now as a part of garbage collection
1598.336526 s: VX_ZONE_WARNING: [vxReleaseContext:1439] Found a reference 0xffff87f21628 of type 00000813 at external count 1, internal count 0, releasing it
1598.336541 s: VX_ZONE_WARNING: [vxReleaseContext:1441] Releasing reference (name=object_array_141) now as a part of garbage collection
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
1598.342111 s: IPC: Deinit ... !!!
1598.342741 s: IPC: DeInit ... Done !!!
1598.342784 s: MEM: Deinit ... !!!
1598.342795 s: DDR_SHARED_MEM: Alloc's: 60 alloc's of 73521264 bytes
1598.342807 s: DDR_SHARED_MEM: Free's : 60 free's of 73521264 bytes
1598.342817 s: DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes
1598.342833 s: MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!
Input Type = Camera;
Input File = IMX219 Camera 0;
Model = /opt/model_zoo/ONR-OD-8420-yolox-s-lite-mmdet-widerface-640x640
Custom Pipeline:
v4l2src device=/dev/video-imx219-cam0 io-mode=5 ! queue leaky=2 ! video/x-bayer, width=1920, height=1080, format=rggb ! tiovxisp sensor-name=SENSOR_SONY_IMX219_RPI dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss.bin format-msb=7 sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a.bin sink_0::device=/dev/v4l-imx219-subdev0 ! video/x-raw, format=NV12 ! tiovxmultiscaler name=split_01 src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1920 src_0::roi-height=1080 target=0 split_01. ! queue ! video/x-raw, width=640, height=640 ! tiovxdlpreproc model=/opt/model_zoo/ONR-OD-8420-yolox-s-lite-mmdet-widerface-640x640 out-pool-size=4 ! application/x-tensor-tiovx ! tidlinferer target=1 model=/opt/model_zoo/ONR-OD-8420-yolox-s-lite-mmdet-widerface-640x640 ! post_0.tensor split_01. ! queue ! video/x-raw, width=1920, height=1080 ! post_0.sink tidlpostproc name=post_0 model=/opt/model_zoo/ONR-OD-8420-yolox-s-lite-mmdet-widerface-640x640 alpha=0.200000 viz-threshold=0.500000 top-N=5 display-model=true ! queue ! mosaic_0. tiovxmosaic name=mosaic_0 target=1 src::pool-size=4 sink_0::startx="<0>" sink_0::starty="<0>" sink_0::widths="<1920>" sink_0::heights="<1080>" ! video/x-raw,format=NV12, width=1920, height=1080 ! queue ! tiperfoverlay main-title=null title="IMX219 Camera 0->ONR-OD-8420-yolox-s-lite-mmdet-widerface-640x640" overlay-type=graph ! tiovxmultiscaler ! video/x-raw, width=1198,height=668 ! queue max-size-buffers=1 ! kmssink driver-name=tidss name="sink" render-rectangle="<222,133,1198,668>" sync=false
APP: Init ... !!!
1609.441644 s: MEM: Init ... !!!
1609.441751 s: MEM: Initialized DMA HEAP (fd=28) !!!
1609.441920 s: MEM: Init ... Done !!!
1609.441936 s: IPC: Init ... !!!
1609.499935 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
1609.505889 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
1609.505956 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
1609.505968 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
1609.505979 s: VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
1609.506554 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0
1609.506835 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1
1609.507058 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2
1609.507215 s: VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3
1609.507237 s: VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!!
1609.507249 s: VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
libtidl_onnxrt_EP loaded 0x28f3fab0
Final number of subgraphs created are : 1, - Offloaded Nodes - 257, Total Nodes - 257
(edgeai-gui-app:1472): GLib-GObject-CRITICAL **: 15:39:03.142: g_object_set_is_valid_property: object class 'GstKMSSink' has no property named 'widget'
Starting video
Stopping video
Removing gst_pipeline
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
1684.978317 s: IPC: Deinit ... !!!
1684.979027 s: IPC: DeInit ... Done !!!
1684.979073 s: MEM: Deinit ... !!!
1684.979084 s: DDR_SHARED_MEM: Alloc's: 14 alloc's of 25407384 bytes
1684.979096 s: DDR_SHARED_MEM: Free's : 14 free's of 25407384 bytes
1684.979107 s: DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes
1684.979122 s: MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!
There is no big difference when using the Camera compared to the Video - it even prints out “Starting Video”/“Stopping Video” similar to when displaying the Video - just nothing appears on the screen.
Screenshots: Video classification
“Custom” window with camera selection:
No Camera window 