[HOWTO] Converting, inferencing and checking YOLO model on Raspberry Pi with HAILO AI HAT

General
1

Hello community,

I’d like to share my experience building a minimal working example for owners of the Raspberry Pi 5 + HAILO AI HAT. This HOWTO covers the following:

  • How to convert a YOLOv8n model into the .HEF format
  • How to run a simple inference on a test image, displaying the results both visually and in the console
  • How to perform a basic comparison of the outputs from both the original and the compiled model

Background

My non-commercial project is focused on object detection and involves guiding a homemade laser to a detected target using real-time camera input. Since this system relies on Raspberry Pi (actually two of them), detection performance has a direct impact on the overall response time.

In an effort to boost FPS, I added the HAILO-8 based AI HAT.
The first logical step was:

  • Run inference with my custom-trained YOLOv8n model using the HAILO chip
  • Compare the detection outputs (bounding boxes, class IDs, confidence scores) against the original model
  • Overlay the results for visual inspection

The Problem

As a result of my own impatience, the lack of structured documentation, and the fatigue that comes from infos being outdated or not matching reality, what seemed like a simple validation task quickly turned into a frustrating experience.

I found myself asking:

  • Which parts of the HAILO software stack are actually required, do I need them at all?
  • Where can I get them, how do I install them, and where exactly should they go?
  • How do I properly convert my YOLOv8n model to .HEF, and what should I do when it throws conversion errors?
  • Is there a simple inference script to compare outputs from the original and compiled models?

Sure, maybe I wasn’t looking in the right places. But between outdated blog posts, contradictory forum advice, and a repo jungle, I ended up installing a pile of useless packages, chasing error messages across modules, and reinventing workarounds that probably already exist.

Eventually, I managed to build a minimal working example — and this post is for anyone who’s trying to do the same, without spending days navigating the same rabbit holes.

If you’re reading this and thinking, “It’s all documented!” — then congrats, you found the right docs. I didn’t.

2 Likes
2

below is a great tutorial

also you need to calibrate your model incase it’s not working with the conversion
1 thing you have to take account on my proj. I found it work better with yolov8s much better detection

1 Like
3

Thanks! That’s really fine tutorial.
But it might be outdated I think. The CUDA/CuDNN part is based on NVidia’s 525 driver. That setup was perfectly working a year ago. After system update I had issues with reinstalling this part and was forced to jongle with versions. Somehow I ended with stable set of 575 driver and CUDA 12.9

It was important for me to get a short and clear inference script without the nested external Python module structure used in the examples repository. It was also important to compare the output numbers from inference, they matter more to me than drawing the boxes, since boxes can be rendered in different ways, while the numbers give a precise understanding of the accuracy of the new inference.
Yolov8N was used because of RPi’s limited computation power. You’re right, probably I should retrain the model with 8s, at least. Thanks for pointing that!

4

use the docker container . I am also using the above guide.
My code here GitHub - k-madsenDK/hailocmdbox: Simpel qt program to control pan and tilt cammera.
is doing the same job wit a pan and tilt camera.
just change the label and put your model.hef instead of yolov11s.hef
This video was my entry to make a yolo model
https://www.youtube.com/watch?v=r0RspiLG260&t=166s

5

sry forgot this raspberry pi 5 with ssd hd and rpi camera is running 30 fps with a system load 25 to 40 % gpu load 10 to 20%