Hi everyone,
I am new to the world of raspberry pi’s, hailo, and running ai models on embedded systems
Currently, I have the raspberry pi5 (8GB version) along with the raspberry pi AI KIT (13 TOPS version, M2 HAT+ Hailo 8L) and the AI camera(Sony IMX 500)
My project involves creating a low-cost real time 3D motion tracking and pose estimation system for artists to be able to use in their performances. In order to optimize the backend, I would like to use yolov12 (came out a day ago), or yolov11
But the following models seem to be unavailable on the hailo model zoo
I was wondering how I could try and integrate these into the pipeline
Additionally, lot of the example demonstrations on the raspberry pi’s website for the AI kit documentation seem to be showing a greater use of the s model variant of the yolo’s
But the model zoo for pose estimation also includes the yolov8 m
So i wanted to gain an understanding of the typical workload and model sizes that my current setup could handle
You can then run the models on your hardware using the HailoRT CLI and compare the numbers.
hailortcli run model.hef
You may see some reduction in FPS because the Raspberry Pi has only a single PCIe lane and may not be as powerful for the pre- and post processing code running on the CPU.
In your application you can see some further reduction in FPS depending on what framework you use (C++, Python, GStremer, OpenCV …).
But this will give you a good comparison and idea what may be possible with your hardware.
Then you can start building your application using the prebuild HEF files. Later you can look into retraining of models or upgrade to the newer models.
If you want to start looking into retraining and converting models. I recommend to install the Hailo AI Software suite Docker and run trough the tutorials first by using the following command inside the Docker.
hailo tutorial
This will start a Jupyter Notebook server with notebooks for each step of the model conversion.
I have some broad clarifying questions and want to provide some broader context for understanding purposes.
I have worked a lot with running AI models on laptops and HPC’s, but not so much embedded systems as mentioned before.
I have been experimenting with the basic pipelines for the hailo-8l as well as the models available for the IMX500 ai camera and comparing the yolo v8 for our “laptop” prototype to the pi prototype
The reason I was asking about the newer models was actually due to the PCIe gen3 bottleneck.
Since the final prototype needs to be integrated with audio output and a functional frontend, the latency needs to be as minimal as possible with the image processing and yolov11 and yolov12 seem to have a lot more architectural improvements, with fewer parameters as well making it potentially faster on the onboard chip.
In your estimate, how long would it take for the pose variants of these models to be integrated into the hailo-model-zoo, since I do not have much experience with the Gstremer pipeline that I see in the sample scripts and would like to prototype as fast as possible.
I was also wondering about the relative latency of python+opencv(which I have worked with a lot more) compared to the gstremer+python framework used in the scripts
And finally, I notice that for integrating the AI kit, the camera and the hailo-8l seem to be operating largely independent of each other. Do you have qny specific guidance on integrating the Sony IMX 500 camera feeds directly into the Hailo pipeline?
And furthermore, if resources become a limitation, how many HAT’s is it possible to add on top of the current rpi5 setup