I posted a couple days back regarding an issue with my model and unexpected output. It turns out that Hailo does not support the ONNX Slice operation yet and they worked around this in their model by removing the handful of layers that were reliant on it and replacing it with a Split operation. So I did this to my model as well. It eliminated some other errors I was having regarding the Slice Op.
However, I am still getting an unsupported model error:
hailo_sdk_client.model_translator.exceptions.ParsingWithRecommendationException: Parsing failed. The errors found in the graph are:
UnsupportedModelError in op /model.22/Add_1: In vertex /model.22/Add_1_input the contstant value shape (1, 2, 8400) must be broadcastable to the output shape [1, 8400, 2]
UnsupportedModelError in op /model.22/Sub: In vertex /model.22/Sub_input the contstant value shape (1, 2, 8400) must be broadcastable to the output shape [1, 8400, 2]
Please try to parse the model again, using these end node names: /model.22/dfl/Reshape_1
I have verified that my model will indeed work by running the ONNX file through both my code and Zetane. So I believe there is some slight difference between my ONNX model and Hailo’s that isn’t seemingly structural.
Does anyone have any insight on why these constants aren’t broadcastable when Hailo’s are?
PS. The error message has constant mispelled as contstant
So in my infinite wisdom I have not tried to run the yolov8n.onnx provided by the Hailo model zoo through the har converter. It turns out that it also has the same issue.
Hi @steven.nix,
Can you further explain what you tried?
Basically you are correct - the Hailo SW doesn’t support some postprocessing operation, and in that case we set the end nodes to be before the unsupported layers.
For yolov8, there are 6 end nodes which are the which are the convolution layers just before the start of the postprocessing ops.
I am using the runner.translate_onnx_model where runner=ClientRunner(hw_arch=“hailo8”).
I also successfully matched my model to the yolov8n.onnx provided by hailo through the model zoo via the yolov8n.zip which contains yolov8n.onnx and yolov8n_nms_config.json.
It seems like I may be able to adjust it further and remove everything to the reshape right before the split which will give the output 1x4x8400 then input that into the final concat with the 1x2x8400 from the sigmoid which will give the final 1x6x8400 im looking for (I only have 2 classes in my model). However I’m not exactly sure what is happening at the end of the model with the Adds, Subs, and Div, operations. It almost seems that it may adjusting the bbox coordinates to xc, yc, w, h.
However if you have a better idea or a different way of doing it let me know and I’ll go down that path as well.
I will go off of the yolov8n.onnx provided by Hailo. At the moment I am not setting either end or start nodes. However I am thinking I should be able to reshape the architecture and feed the output from “/model.22/dfl/Reshape_1_output_0” directly to /model.22/Concat_4 and then use the runner.translate_onnx_model. Im not sure if the output will be correct though.
I could also grab the 6 convolution layers directly from the head which is what I believe you said Hailo is doing.
Ok so I made the changes I mentioned and successfully converted it to the har format and was able to run it. That being said I still want to give the Hailo solution a try. Where is the source code or the yaml file or whatever is needed where Hailo is basically attaching their own solution to the head?
Also here are pictures of the Hailo Yolov8 architecture vs the changes I made.
Ok I see what is going on. After reviewing the yolov8n model provided in the model zoo using hailomz parse yolov8n I see what you are talking about. When you mention setting the end nodes after the the CNN Layers in the head.
[info] Translation started on ONNX model yolov8n
[info] Restored ONNX model yolov8n (completion time: 00:00:00.03)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.13)
[info] NMS structure of yolov8 (or equivalent architecture) was detected.
[info] In order to use HailoRT post-processing capabilities, these end node names should be used: /model.22/cv2.0/cv2.0.2/Conv /model.22/cv3.0/cv3.0.2/Conv /model.22/cv2.1/cv2.1.2/Conv /model.22/cv3.1/cv3.1.2/Conv /model.22/cv2.2/cv2.2.2/Conv /model.22/cv3.2/cv3.2.2/Conv.
[info] Start nodes mapped from original model: 'images': 'yolov8n/input_layer1'.
[info] End nodes mapped from original model: '/model.22/cv2.0/cv2.0.2/Conv', '/model.22/cv3.0/cv3.0.2/Conv', '/model.22/cv2.1/cv2.1.2/Conv', '/model.22/cv3.1/cv3.1.2/Conv', '/model.22/cv2.2/cv2.2.2/Conv', '/model.22/cv3.2/cv3.2.2/Conv'.
[info] Translation completed on ONNX model yolov8n (completion time: 00:00:00.47)
So then my question becomes where is the source code that is handling the output from these convolutional layers by Hailo. If I have that then I believe pretty much all my problems are solved.
I have now extracted and looked over the HAR file and realize that the original model has had the end nodes set to be the convolutional layers mentioned. However, It looks like the post-processing is then in its own ONNX file and the meta data links the output from the Head to the yolov8n.postprocess.onnx.
It turns out that when at least dealing with the translate_onnx_model method, it can automatically separate the post processing and include it in the HAR file if you set the right end nodes.
It also looks like it can potentially detect the correct end nodes after running the the translate method if no other issues occur.
However it blends in with the rest of the info logs:
[info] Translation started on ONNX model sorted_modified_nano_best
[info] Restored ONNX model sorted_modified_nano_best (completion time: 00:00:00.03)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.12)
[info] NMS structure of yolov8 (or equivalent architecture) was detected.
[info] In order to use HailoRT post-processing capabilities, these end node names should be used: /model.22/cv2.0/cv2.0.2/Conv /model.22/cv3.0/cv3.0.2/Conv /model.22/cv2.1/cv2.1.2/Conv /model.22/cv3.1/cv3.1.2/Conv /model.22/cv2.2/cv2.2.2/Conv /model.22/cv3.2/cv3.2.2/Conv.
[info] Start nodes mapped from original model: 'images': 'sorted_modified_nano_best/input_layer1'.
[info] End nodes mapped from original model: '/model.22/cv2.0/cv2.0.2/Conv', '/model.22/cv3.0/cv3.0.2/Conv', '/model.22/cv2.1/cv2.1.2/Conv', '/model.22/cv3.1/cv3.1.2/Conv', '/model.22/cv2.2/cv2.2.2/Conv', '/model.22/cv3.2/cv3.2.2/Conv'.
[info] Translation completed on ONNX model sorted_modified_nano_best (completion time: 00:00:00.36)
[info] Saved HAR to:
After I corrected the unsupported Slice ops and matched my model to Hailo’s I got a HAR file that finally matches theirs.
I highly recommend displaying the HailoRT message as a warning or something that will pop much better.
[info] In order to use HailoRT post-processing capabilities, these end node names should be used: /model.22/cv2.0/cv2.0.2/Conv /model.22/cv3.0/cv3.0.2/Conv /model.22/cv2.1/cv2.1.2/Conv /model.22/cv3.1/cv3.1.2/Conv /model.22/cv2.2/cv2.2.2/Conv /model.22/cv3.2/cv3.2.2/Conv.
That being said this trail of issues were caused overall by my lack of understanding. Thank you for your support @Omer
I was reading the Dataflow Guide and it seems to insinuate that when the HAR file is compiled to a HEF the metadata linking to the postprocessing in the HAR file along side any Model Script files also applied to the model take effect. Meaning that I should see the expected 1x6x8400 output from my HEF file instead of the 6 outputs I am seeing from my HAR file even though it seems the post processing is included.
The postprocessing for yolov8 can be added to the compiled model by adding the nms_postprocess command to the model script.
You can see how this is done here, and read more about the command here.
Hi @steven.nix@Omer based on my understading the output node you set in to reshape_1 so by this the mathematical operation has been omitted but does that mean the model will be hanging to the /model.22/dfl/Reshape_1_output_0 as the output node . Based on your diagram there are concatenation on the model architecture. What is there is a multiple split does that mean it will omit that part aswell.
I actually didn’t follow through with that version of the model and instead used the original architecture and split the model after the detection head like in the Hailo examples. I then provided a hailo script that then connects the detection head to the NMS processing when the model is optimized and then should be included in the compiled hef. The hailo script is mentioned in the hailo dataflow compiler user guide. However I believe the model zoo has an existing .alls script specifically for yolov8 although you may have to find it.
