Trying to extend a basic cpp object detection example from Hailo-Application-Code-Examples

[mikepwright]

Hi everyone,

I need some debugging help. I’ve retrained both yolov8x and yolov8s against custom dataset. All work well with inferencing tests using sample images and a new weights file.

I have converted my weights PT file to ONNX
I have successfully compiled my ONNX file into an HEF

hailomz compile --ckpt=./birds-yolov8s.onnx --hw-arch hailo8l --calib-path data/val/ --classes 1011 --model-script yolov8s.alls --yaml yolov8s.yaml

Note I did not change much from the provided alls, yaml, and json config files. However I did touch up the number of classes to 1011, and added model_optimization_config(calibration, batch_size=1) to alls because I was running out of memory on GPU compiling.

Next, I moved my model over to RPI5 with 8L accelerator.
I compiled Hailo-Application-Code-Examples/runtime/cpp/object_detection/general_detection_inference without changes
Then, I run using
build/x86_64/vstream_detection_example_cpp -hef=birds-yolov8s.hef -input=Cardinal.jpg -num=200

The process runs but without meaningful/accurate detection results.

Tail end of results before ending:

-I- Inference finished successfully

-I-----------------------------------------------
-I- birds-yolov8s
-I-----------------------------------------------

-I-----------------------------------------------
-I- Inference & Postprocess
-I-----------------------------------------------
-I- Average FPS:  36.71
-I- Total time:   5.45 sec
-I- Latency:      27.24 ms
-I-----------------------------------------------

What additional details could be helpful to understand my attempt? I am struggling to see a successful inference result.

[omria]

Hey @mikepwright,

For initial debugging, could you please try these steps:

1. Run hailortcli run {hef} with your model file
2. Execute hailomz or hailo(dfc) parse-hef your_model.hef to analyze the HEF structure

If your model uses hailortPP, you might want to try running it using the Raspberry Pi examples as well.

Let me know what results you get from these diagnostic steps.

[mikepwright]

I ran both of these on the RPi5

(venv) edge-admin@outdoor-edge:~/Hailo-Application-Code-Examples/runtime/cpp/object_detection $ hailortcli run birds-yolov8s.hef
Running streaming inference (birds-yolov8s.hef):
Transform data: true
Type: auto
Quantized: true
Network yolov8s/yolov8s: 100% | 185 | FPS: 36.83 | ETA: 00:00:00

Inference result:
Network group: yolov8s
Frames count: 185
FPS: 36.83
Send Rate: 362.08 Mbit/s
Recv Rate: 2686.13 Mbit/s

(venv) edge-admin@outdoor-edge:~/Hailo-Application-Code-Examples/runtime/cpp/object_detection $ hailo parse-hef birds-yolov8s.hef
(hailo) Running command ‘parse-hef’ with ‘hailortcli’
Architecture HEF was compiled for: HAILO8L
Network group name: yolov8s, Multi Context - Number of contexts: 3
Network name: yolov8s/yolov8s
VStream infos:
Input yolov8s/input_layer1 UINT8, NHWC(640x640x3)
Output yolov8s/yolov8_nms_postprocess FLOAT32, HAILO NMS BY CLASS(number of classes: 1011, maximum bounding boxes per class: 100, maximum frame size: 2026044)
Operation:
Op YOLOV8
Name: YOLOV8-Post-Process
Score threshold: 0.200
IoU threshold: 0.70
Classes: 1011
Cross classes: false
NMS results order: BY_CLASS
Max bboxes per class: 100
Image height: 640
Image width: 640

[mikepwright]

Hi @omria

Quick update from my end.

I went through the whole process again starting with retraining yolov8s using my custom dataset and made sure I am on all the latest versions of Hailo.

Currently I am hitting the same issue mentioned in this post: hailo_sdk_client.tools.core_postprocess.nms_postprocess.NMSConfigPostprocessException: The layer yolov8n/conv41 doesn't have one output layer

I noticed some of the instructions may be outdated now due to changes. For example it seems --no-nms is no longer an option with the compile command.

I guessed that removing this line from a copy of yolov8s.alls might be equivalent?

nms_postprocess(“…/…/postprocess_config/yolov8s_nms_config.json”, meta_arch=yolov8, engine=cpu)

Removing this allows the compile to proceed without the error about conv42. Couple questions:

1. I am specifying a list of end-node-names per the parse but I still get the error. Why does the error still occur even after supplying the end-node-names?

2. If compile succeeds without nms post processing, what do I do next to make sure this will create a functional HEF? I am assuming it won’t have bounding box and class identification in the output without NMS pos-process?

3. You mentioned in the referenced post that you can manually post-process after inference. Can you point me to any CPP code examples of this?

Thank you so much for your help! I feel I might be within inches of success!

[shashi]

Hi @mikepwright
If you are able to compile without postprocess included, the postprocessors needs to run outside. Our PySDK package can help with this. If you are ok to try our PySDK, I can provide you the steps involved.

[mikepwright]

yes, I was able to compile to HEF. For testing the model, I had switched to C++ because I was getting some kind of OOM error on the RPI5 with Python but if you have an end to end example using PySDK I can give it a try. Thanks @shashi

[shashi]

Hi @mikepwright
You can follow the steps outlined in our user guide: User Guide 3: Simplifying Object Detection on a Hailo Device Using DeGirum PySDK

[mikepwright]

@shashi

I am running into the same issue discussed with another user where degirum is not able to local my model

In my case, I am using yolov8s with my custom dataset. Does this method only support yolov11n?

I see there are specifics about naming conventions. I dropped ‘v’ in the naming used in my model zoo however it still is unable to locate my model.

Here are the contents of my json file:

Blockquote
{
“ConfigVersion”: 10,
“DEVICE”: [
{
“DeviceType”: “HAILO8”,
“RuntimeAgent”: “HAILORT”,
“SupportedDeviceTypes”: “HAILORT/HAILO8”
}
],
“PRE_PROCESS”: [
{
“InputType”: “Image”,
“InputN”: 1,
“InputH”: 640,
“InputW”: 640,
“InputC”: 3,
“InputPadMethod”: “letterbox”,
“InputResizeMethod”: “bilinear”,
“InputQuantEn”: true
}
],
“MODEL_PARAMETERS”: [
{
“ModelPath”: “yolo8s.hef”
}
],
“POST_PROCESS”: [
{
“OutputPostprocessType”: “Detection”,
“PythonFile”: “HailoDetectionYolo.py”,
“OutputNumClasses”: 80,
“LabelsPath”: “labels_coco.json”,
“OutputConfThreshold”: 0.3
}
]
}

[shashi]

Hi @mikepwright
Please see the comment here: User Guide 3: Simplifying Object Detection on a Hailo Device Using DeGirum PySDK - Guides - Hailo Community. You need to add one more field to the JSON called CheckSum.

[mikepwright]

@shashi Just a quick update… I have been able to successfully compile and run detections with my custom yolo11n weights. This article together with yours input helped me generate the working HEF. Thank you!

Guide to using the DFC to convert a modified YoloV11 on Google Colab

This guide will walk you through the end to end process of how to retrain and compile YoloV11 from a PyTorch file into a HEF file executable by the Hailo accelerators using Google Colab. I decided to use colab as it’s a managed environment and should be accesible to everyone, although this does introduce some complexities. This also works for other Yolo versions, keep in mind that YoloV8 through YoloV11 use “equivalent architectures” at least in the eyes of the DFC. Preparing our dataset: Yolo …

Now on to packaging it all up and managing it with Nomad!