Accelerated Real-Time Commercial Detection and Replacement on BeagleY-AI
The existing commercial detection project using bbai-64 implements a chunk-based commercial classifier using a quantized TFLite model with the TIDL delegate for inference. While NPU acceleration is already used, the overall system is built around an OpenCV-based pipeline, making it unsuitable for real-time operation and live video streams on BeagleY-AI.
This project proposes a BeagleY-AI first re-architecture of the commercial detection system, focused on real-time streaming, Edge AI acceleration, and hardware-aware multimedia design. Rather than extending the existing OpenCV workflow, the system will be rebuilt around GStreamer pipelines and NPU-accelerated inference, targeting live HDMI or network video inputs.
Core Idea
Design and implement a real-time commercial detection and replacement system on BeagleY-AI by:
- Retaining quantized neural networks executed via TFLite or ONNX-based TIDL export for NPU inference.
- Moving all video handling from OpenCV to GStreamer-based streaming pipelines.
- Eliminating CPU-bound frame processing from the critical path.
- Supporting live video input sources (USB HDMI capture devices, network streams).
- Providing configurable replacement behavior (blackout, alternate content, or audio masking).
This makes the system suitable for continuous live video streams and ensures that it is architected around BeagleY-AI’s NPU and multimedia hardware, rather than desktop-style batch processing.
Hardware Skills
- Understanding of BeagleY-AI hardware
- Embedded Linux systems
- Basic multimedia concepts
Software Skills
- Python and/or C++
- GStreamer multimedia frameworks
- Neural network quantization and deployment
- Embedded performance optimization
Expected Outcomes
- A GStreamer-based real-time commercial detection pipeline for BeagleY-AI
- Quantized models running on the NPU
- Live HDMI or network video input support
- Target sustained throughput (≥20 FPS) on BeagleY-AI.
Expected Mentors
@jkridner @lorforlinux
Expected Size of Project:
Medium (≈350 hours)