Raspberry Pi 5 + Hailo8L for instance segmentation using yolo v5

General
1

I am seeing this error while converting the yolov5m-seg model (in onnx format) to .hef using hailomz command

hailo_sdk_client.model_translator.exceptions.MisspellNodeError: Unable to find end node names: [‘output1’, ‘Conv_326’, ‘Conv_305’, ‘Conv_284’], please verify and try again.

Steps followed so far :
I have re-trained the yolov5m-seg.pt using a custom annotated dataset and verified its working in an ubuntu 22.04 PC. Then converted this model into onnx format using the python script

from ultralytics import YOLO
model = YOLO(“runs/train-seg/exp4/weights/best.pt”)
model.export(format=“onnx”, imgsz=640, opset=11)

I have followed this example for exporting
Retraining example

after this, I have tried to convert onnx into .hef file using the below command

hailomz compile yolov5m_seg --ckpt=retrained_model.onnx --hw-arch hailo8l --calib-path dataset/train/images --classes 1

This resulted in the below error

Start run for network yolov5m_seg …
Initializing the hailo8l runner…
[info] Translation started on ONNX model yolov5m_seg
[info] Restored ONNX model yolov5m_seg (completion time: 00:00:00.39)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:01.39)
Traceback (most recent call last):
…
…
…
raise MisspellNodeError(f"Unable to find {err_str}: {wrong_names}, please verify and try again.")
hailo_sdk_client.model_translator.exceptions.MisspellNodeError: Unable to find end node names: [‘output1’, ‘Conv_326’, ‘Conv_305’, ‘Conv_284’], please verify and try again.

please help me navigate through this [I have successfully converted the yolo v8 model into .hef file for the same segmentation task. but the postprocessing was not optimized for yolov8-seg, so I had to fall back to yolov5-seg]

2

Hey @mailtoprasan,

I noticed that the issue you’re experiencing is related to the incorrect output node names in your model. It seems that the output node names are not matching the ones specified in the compilation command, possibly because the model has been modified.

To resolve this, I suggest the following steps:

  1. Open your model file (retrained_model.onnx) in Netron (https://netron.app) to inspect the model architecture.
  2. In Netron, navigate to the “Model Properties” section on the left sidebar and expand it if necessary.
  3. Look for the “Outputs” subsection within “Model Properties” and take note of the names listed under it. These are the current output node names of your model.
  4. Update the --output-layer argument in the hailomz compile command or in the alls configuration file with the correct output node names you found in Netron.

If you have any further questions or if you encounter any problems while following these steps, please let me know, and I’ll be happy to assist you further.

1 Like
4

Thank you @omria for the suggestion. Recompiling with end-node-names solved this problem for me!

Steps followed :

I have opened the onnx model in netron app. It showed the output names as following

outputnames
outputnames483×396 26.1 KB

yolov5m_output
yolov5m_output1724×866 118 KB

I have updated hailomz compile arguments to include end-node-names as below

hailomz compile yolov5m_seg --ckpt=retrained_model.onnx --hw-arch hailo8l --calib-path dataset/train/images --classes 1  --end-node-names output0 465 onnx::Split_480 onnx::Split_518 onnx::Split_556

This resolved the earlier error of not finding the output node names, but still unable to complete the compilation siting dimension mismatch

<Hailo Model Zoo INFO> Start run for network yolov5m_seg ...
<Hailo Model Zoo INFO> Initializing the hailo8l runner...
[info] Translation started on ONNX model yolov5m_seg
[info] Restored ONNX model yolov5m_seg (completion time: 00:00:00.39)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:01.25)
[info] Simplified ONNX model for a parsing retry attempt (completion time: 00:00:02.48)
Traceback (most recent call last): 
.....
.....
.....
hailo_sdk_client.model_translator.exceptions.ParsingWithRecommendationException: Parsing failed. The errors found in the graph are:
 UnsupportedShuffleLayerError in op /model.24/Reshape_1: Failed to determine type of layer to create in node /model.24/Reshape_1
 UnsupportedModelError in op /model.24/Add: In vertex /model.24/Add_input the constant value shape (1, 3, 80, 80, 2) must be broadcastable to the output shape [80, 80, 6]
 UnsupportedModelError in op /model.24/Mul_3: In vertex /model.24/Mul_3_input the constant value shape (1, 3, 80, 80, 2) must be broadcastable to the output shape [80, 80, 6]
 UnsupportedShuffleLayerError in op /model.24/Reshape_3: Failed to determine type of layer to create in node /model.24/Reshape_3
 UnsupportedModelError in op /model.24/Add_1: In vertex /model.24/Add_1_input the constant value shape (1, 3, 40, 40, 2) must be broadcastable to the output shape [40, 40, 6]
 UnsupportedModelError in op /model.24/Mul_7: In vertex /model.24/Mul_7_input the constant value shape (1, 3, 40, 40, 2) must be broadcastable to the output shape [40, 40, 6]
 UnsupportedShuffleLayerError in op /model.24/Reshape_5: Failed to determine type of layer to create in node /model.24/Reshape_5
 UnsupportedModelError in op /model.24/Add_2: In vertex /model.24/Add_2_input the constant value shape (1, 3, 20, 20, 2) must be broadcastable to the output shape [20, 20, 6]
 UnsupportedModelError in op /model.24/Mul_11: In vertex /model.24/Mul_11_input the constant value shape (1, 3, 20, 20, 2) must be broadcastable to the output shape [20, 20, 6]
Please try to parse the model again, using these end node names: /model.24/proto/cv3/act/Mul, /model.24/Transpose_1, /model.24/Transpose, /model.24/Transpose_2

When the end-node-names was changed as suggested in the error message like below

hailomz compile yolov5m_seg --ckpt=retrained_model.onnx --hw-arch hailo8l --calib-path Dataset/train/images --classes 1 --end-node-names /model.24/Transpose /model.24/Transpose_2 /model.24/Transpose_1 /model.24/proto/cv3/act/Mul

Compilation was successful and generated .hef file

To test this .hef file, I have put it in a raspberry pi 5 with hailo8L

  1. Tried replacing this .hef file in the hailo-rpi5-examples/doc/basic-pipelines.md at main · hailo-ai/hailo-rpi5-examples · GitHub
    but it gave segmentation fault
  2. Tried to use it in the Hailo-Application-Code-Examples/runtime/python/instance_segmentation at main · hailo-ai/Hailo-Application-Code-Examples · GitHub
    but post processing was not successful

Is this the right way to test?