YOLOv5 object detection on BB AI-64: end-to-end walkthrough

Not a question, just a PSA.

I’ve been experimenting with training models for TIDL. As far as I have been able to tell, they have no end-to-end documentation on this process. The various sample repos are omnibus “do-it-all” scripts, but aren’t practical for personal reference.

I have put together a collection of scripts and associated notes which is minimal and self-contained. It goes through the process of training a YOLOv5 detection model, compiling it with the TIDL tools, and running it on a BB AI-64.

I’d be happy to discuss my learnings or provide help where I can. Feel free to open issues or comment here with questions or suggestions. Everything I’ve written is based on experimentation, so if anything is wrong or could be improved please do let me know.

The code and documentation is here: GitHub - WasabiFan/tidl-yolov5-custom-model-demo: A walkthrough and personal notes on running YOLOv5 models with TIDL.


Great work @kaelinl

Btw, what is the fps are you achieving ?

Best regards and congratz

I’ve been running the smallest models, so yolov5s6. I’m timing from Python (run inference in a loop 2000 times, divide wall clock time taken by 2000). At 640x640 I was getting an average of 15ms and at 384*384 it’s around 7.5ms. I haven’t tried C++ to see closer to native performance. And as I note in the README, the number of detections affects the time taken due to postprocessing; I’ve seen a swing of 1ms or so due to this.

PSA: I’ve updated the samples to include a C++ inference app. It mirrors the Python implementation, printing the time per forward pass and drawing boxes on images. I used onnxruntime, the same API as is available in Python.

There is also a “tidlrt” available in C++, which may be faster, but I haven’t tried. Since it’s TI’s proprietary API it’s probably harder to use; at a glance, it doesn’t look pleasant.