Run custom YOLOv8 Model on RPI5 with Hailo AI Shield

General
, ,
1

Hello Hailo-Community,

I am working with the Hailo-Platform for 2 Weeks now and reached the Point where I want to deploy my custom model on the Hailo8-Device for inference.

My setup:

  • YOLOv8s Model for Object detection and classification
  • Converted to .onnx using Ultralytics
  • Converted to .har and then .hef using Hailo Dataflow Compiler
  • Running the model with degirum

What I have tried:

  • I checked if my degirum pipeline is working with the model from the hailo examples in the DeGirum git
  • This runs through and gives detections in the output
  • I changed the model to my custom YOLOv8s model and i get this Error: degirum.exceptions.DegirumException: Failed to perform model ‘yolov8s_quantized_model’ inference: Model ‘yolov8s_quantized_model’ inference failed: [ERROR]Incorrect value of parameter
    Hailo Async Runtime: Input DG Tensor type DG_FLT does not match Hailo input type DG_UINT8
    hailo_plugin_impl.cpp: 136 [{anonymous}::to_hailo_tensor]
    When running model ‘yolov8s_quantized_model’

What i found out so far:

When i check the models Architecture of the model from the Degirum git it looks like this:
Architecture HEF was compiled for: HAILO8
Network group name: yolov8n_relu6_coco, Single Context
Network name: yolov8n_relu6_coco/yolov8n_relu6_coco
VStream infos:
Input yolov8n_relu6_coco/input_layer1 UINT8, NHWC(640x640x3)
Output yolov8n_relu6_coco/conv47 UINT8, FCR(80x80x64)
Output yolov8n_relu6_coco/conv59 UINT8, FCR(40x40x64)
Output yolov8n_relu6_coco/conv70 UINT8, FCR(20x20x64)
Output yolov8n_relu6_coco/conv48 UINT8, FCR(80x80x80)
Output yolov8n_relu6_coco/conv60 UINT8, FCR(40x40x80)
Output yolov8n_relu6_coco/conv71 UINT8, FCR(20x20x80)

If I check my custom model it looks like this:
Architecture HEF was compiled for: HAILO8
Network group name: yolov8s, Multi Context - Number of contexts: 2
Network name: yolov8s/yolov8s
VStream infos:
Input yolov8s/input_layer1 UINT8, F8CR(640x640x3)
Output yolov8s/yolov8_nms_postprocess FLOAT32, HAILO NMS BY CLASS(number of classes: 2, maximum bounding boxes per class: 100, maximum frame size: 4008)
Operation:
Op YOLOV8
Name: YOLOV8-Post-Process
Score threshold: 0.250
IoU threshold: 0.45
Classes: 2
Max bboxes per class: 100
Image height: 640
Image width: 640

Only mayor difference I have noticed is the format of the input Tensor being F8CR instead of NHWC. I dont know if that causes the issue.

My .json file for the Model looks like this:
{
“Checksum”: “d6c4d0b9620dc2e5e215dfab366510a740fe86bf2c5d9bd2059a6ba3fe62ee63”,
“ConfigVersion”: 10,
“DEVICE”: [
{
“DeviceType”: “HAILO8”,
“RuntimeAgent”: “HAILORT”,
“SupportedDeviceTypes”: “HAILORT/HAILO8”,
“ThreadPackSize”: 6
}
],
“MODEL_PARAMETERS”: [
{
“ModelPath”: “yolov8s_quantized_model.hef”
}
],
“POST_PROCESS”: [
{
“LabelsPath”: “orientation_labels.json”,
“OutputNumClasses”: 2,
“OutputPostprocessType”: “DetectionYoloV8”
}
],
“PRE_PROCESS”: [
{
“InputN”: 1,
“InputH”: 640,
“InputW”: 640,
“InputC”: 3,
“InputQuantEn”: true
}
]
}

I hope you can help me out with this.

Thank you in advance and best regards,
Simon

2

Hi @Simon_Wolf
Welcome to the Hailo community. The reason you are seeing this error is due to the fact that your model was compiled to use in-built NMS of Hailo. In PySDK, we support our own postprocessor and Hailo NMS as well. But the JSON needs to be configured accordingly. The instructions to do this are covered in User Guide 2: Running Your First Object Detection Model on a Hailo Device Using DeGirum PySDK. Briefly, you need to use a different postprocessing module. Please try this and let us know if you still see issues.

3

Hi @shashi,
thank you for your reply. Could have seen that myself, sorry. I´ll try a post processor for the Hailo NMS.

I´ll come back to this after I tried it and give you feedback.

4

I changed the post process module to this now:
“POST_PROCESS”: [
{
“OutputPostprocessType”: “None”
}

But i still get this error:

Traceback (most recent call last):
File “/usr/lib/python3.11/runpy.py”, line 198, in _run_module_as_main
return _run_code(code, main_globals, None,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/usr/lib/python3.11/runpy.py”, line 88, in _run_code
exec(code, run_globals)
File “/home/test/.vscode/extensions/ms-python.debugpy-2025.8.0-linux-arm64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/main.py”, line 71, in
cli.main()
File “/home/test/.vscode/extensions/ms-python.debugpy-2025.8.0-linux-arm64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py”, line 501, in main
run()
File “/home/test/.vscode/extensions/ms-python.debugpy-2025.8.0-linux-arm64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py”, line 351, in run_file
runpy.run_path(target, run_name=“main”)
File “/home/test/.vscode/extensions/ms-python.debugpy-2025.8.0-linux-arm64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py”, line 310, in run_path
return _run_module_code(code, init_globals, run_name, pkg_name=pkg_name, script_name=fname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/test/.vscode/extensions/ms-python.debugpy-2025.8.0-linux-arm64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py”, line 127, in _run_module_code
_run_code(code, mod_globals, init_globals, mod_name, mod_spec, pkg_name, script_name)
File “/home/test/.vscode/extensions/ms-python.debugpy-2025.8.0-linux-arm64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py”, line 118, in _run_code
exec(code, run_globals)
File “/home/test/Desktop/application/inference_hailo.py”, line 32, in
inference_result = model(img_batched)
^^^^^^^^^^^^^^^^^^
File “/home/test/Desktop/application/hailo_venv/lib/python3.11/site-packages/degirum/log.py”, line 59, in wrap
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File “/home/test/Desktop/application/hailo_venv/lib/python3.11/site-packages/degirum/model.py”, line 234, in call
return self.predict(data)
^^^^^^^^^^^^^^^^^^
File “/home/test/Desktop/application/hailo_venv/lib/python3.11/site-packages/degirum/log.py”, line 59, in wrap
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File “/home/test/Desktop/application/hailo_venv/lib/python3.11/site-packages/degirum/model.py”, line 225, in predict
res = list(self._predict_impl(source))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/home/test/Desktop/application/hailo_venv/lib/python3.11/site-packages/degirum/model.py”, line 1230, in _predict_impl
raise DegirumException(msg) from saved_exception
degirum.exceptions.DegirumException: Failed to perform model ‘yolov8s_quantized_model’ inference: Model ‘yolov8s_quantized_model’ inference failed: [ERROR]Functionality is not supported
Hailo Async Runtime: Unsupported format order in to_hailo_tensor
hailo_plugin_impl.cpp: 164 [{anonymous}::to_hailo_tensor]
When running model ‘yolov8s_quantized_model’

FYI I am using HailoRT-CLI version 4.21.0

5

Hi @Simon_Wolf
Now, it does look like the input tensor format is the problem. Can you share your compiler settings? Generally, input format is NHWC.

6

I recently worked on the same setup as yours (custom hef, yolov5 & v8) and wrote this tutorial: Custom dataset with Hailo AI Hat, Yolo, Raspberry PI 5, and Docker | by Luiz doleron | Luiz d'Oleron | Apr, 2025 | Towards AI
and this tutorial: Deploying a custom YOLOv8 model on Raspberry Pi 5 with Hailo AI Hat+

7

Hi toghether,
I used the approach of @Luiz_doleron and converted my model again. Now i have the correct input tensor format NHWC and the inference is running through.

Thank you all for your help. Now i just have to figure out how to display the inference results :slight_smile:

8

Hi @Simon_Wolf
Glad to see you got the model working. Displaying inference results using PySDK is very simple (a single line of code): display(inference_result.imageOverlay). You can see examples at DeGirum/hailo_examples: DeGirum PySDK with Hailo AI Accelerators or User Guide 3: Simplifying Object Detection on a Hailo Device Using DeGirum PySDK - Guides - Hailo Community

9

Thank you guys very much. I got everything up and running and now am a proud owner of my custom YOLOv8 inference running on a Hailo8 :slight_smile:

PS: here is the proof

image
image505×585 60.8 KB

3 Likes
10

Congratulations! Well done. Thank you for sharing.