UnsupportedShuffleLayerError when using Reshape node in ONNX

General
, ,
1

Hi there,
I get the following error when I try to convert an ONNX model to .har using the hailo Parsing tutorial.

hn, npz = runner.translate_onnx_model(
    onnx_path,
    onnx_model_name,
    start_node_names=["input"],
    end_node_names=["output"],
    net_input_shapes={"input": [1,3,224,224]},
)```
 
![Screenshot from 2024-09-12 10-18-26|690x325](upload://lCzWKl0946NtSdJdBj9BPEJ2LLy.png)

ParsingWithRecommendationException: Parsing failed. The errors found in the graph are:
UnsupportedShuffleLayerError in op /model.105/Reshape: Failed to determine type of layer to create in node /model.105/Reshape
UnsupportedShuffleLayerError in op /model.105/Transpose: Failed to determine type of layer to create in node /model.105/Transpose
UnsupportedFeatureSplitterLayerError in op /model.105/Split: Feature splitter vertex /model.105/Split is splitting input over unsupported axis 4
UnsupportedShuffleLayerError in op /model.105/Reshape_2: Failed to determine type of layer to create in node /model.105/Reshape_2
UnsupportedShuffleLayerError in op /model.105/Reshape_3: Failed to determine type of layer to create in node /model.105/Reshape_3
UnsupportedShuffleLayerError in op /model.105/Transpose_1: Failed to determine type of layer to create in node /model.105/Transpose_1
UnsupportedFeatureSplitterLayerError in op /model.105/Split_1: Feature splitter vertex /model.105/Split_1 is splitting input over unsupported axis 4
UnsupportedShuffleLayerError in op /model.105/Reshape_5: Failed to determine type of layer to create in node /model.105/Reshape_5
UnsupportedShuffleLayerError in op /model.105/Reshape_6: Failed to determine type of layer to create in node /model.105/Reshape_6
UnsupportedShuffleLayerError in op /model.105/Transpose_2: Failed to determine type of layer to create in node /model.105/Transpose_2
UnsupportedFeatureSplitterLayerError in op /model.105/Split_2: Feature splitter vertex /model.105/Split_2 is splitting input over unsupported axis 4
UnsupportedShuffleLayerError in op /model.105/Reshape_8: Failed to determine type of layer to create in node /model.105/Reshape_8
Please try to parse the model again, using these end node names: /model.105/m.0/Conv, /model.105/m.2/Conv, /model.105/m.1/Conv```

2

Screenshot from 2024-09-12 10-18-26
Screenshot from 2024-09-12 10-18-261897×896 146 KB

3

We used the following repo to train our model in Pytorch first, than converted to ONNX.

4

I have also tried parsing a similar model with the Network Name “yolov7” which I downloaded from the Model explorer:
Model Zoo by Hailo | AI Model Explorer to Find The Best NN Model

I set the architecture as “hailo8l” and use the parser to convert from onnx to .haf - Yet I get a similar error:

ParsingWithRecommendationException: Parsing failed. The errors found in the graph are:
 UnsupportedShuffleLayerError in op Reshape_303: Failed to determine type of layer to create in node Reshape_303
 UnsupportedShuffleLayerError in op Transpose_304: Failed to determine type of layer to create in node Transpose_304
Please try to parse the model again, using these end node names: Conv_302
5

Please have a look at the last line of the error message. You will need to use the end node names suggested in the error message not “output”.

You will need to ensure you get all outputs of the model not just Conv_302.

Here are a few more pointers to get a better understanding.

Tutorial

Please go trough the tutorial on how to convert a model. In the Hailo AI Software suite docker call

hailo tutorial

Note the start and end node names; they are important. Some models contain layers at the beginning and/or end of the model that need to be excluded during the parsing step and implemented on the host.

Model Zoo CLI

Have a look at the model in the Hailo Model Zoo. First use the Model Zoo CLI. Inside the Hailo AI Software suite docker run:

cd /local/shared_with_docker/
hailomz parse yolov7

This will download the original ONNX file and parse it into a har file. You can find the original in the /local/shared_with_docker/.hailomz/model_files/ObjectDetection/ directory.

Now open Netron and open the ONNX and the HAR file.

https://netron.app

You will see that the Reshape and Transpose layers are part of the ONNX but not included in the HAR file.

Model Zoo YAML

The Model Zoo uses YAML files to store the information needed to parse a model. This can be a useful hint on how to parse a model. Have a look at the file for yolov7 and compare it with the ONNX graph in Netron.

Hailo Model Zoo - Yolov7 YAML

You will see the end node names. Note that the end node names can be slightly different when you use a variant from a different source. You can use Netron to determine the names for your specific model.

6

Hi @klausk,

With respect to the original model we tried I understand the error message, the issue is that there are a lot more layers in the .onnx model that follow after the /model.105/m.0/Conv layer (see image

Screenshot from 2024-09-17 16-32-50
Screenshot from 2024-09-17 16-32-501605×918 117 KB

)

w.r.t ^ When you mention the above, does ‘implemented on the host’ mean that we are required to manually add the remaining nodes?

7

The Netron graph can be a bit misleading. The number of layers is not necessarily a reflection of the computation needed. Some boxes require a large amount of computation while in other cases a many boxes together only require very few operations.

AI frameworks have been developed for CPUs and GPUs and not for AI accelerators alone. They can describe operators that you would not want to execute on the Hailo device. They are mostly at the beginning and end of a network. We call it pre- and post-processing. These need to be implemented and executed on the host CPU.

We do provide some of the post processing code for popular networks in Tappas Postprocessing and our Application Code Runtime Examples. In some cases we even run that code under HailoRT. e.g. the NMS for Yolo can be part of the HEF that runs on the host. Have a look at the model script for Yolov7. It contains an instruction to add the NMS.

Hailo Model Zoo - Yolov7 ALLS

nms_postprocess("../../postprocess_config/yolov7_nms_config.json", yolov5, engine=cpu)

In other cases you will need to rewrite the code in C/C++ or Python.

8

Hi @klausk,

I understand that the last layers are post-processing and not heavy convolutions. The issue is that we are looking at moving from the google coral platform to Hailo for our devices. We have 3 models we run in a pipeline on a raspberry pi and are looking to compare over all performance.

It doesn’t suit us right now to manually add layers that can not be converted just for testing purposes. Is there an example of an object detector that we can customise for our images and labels that we can convert the whole model without having to manually add the end layers? To create the model we are discussing I used the hailo supported yolo7 with Transferlearning and out custom labels and datasets (GitHub - WongKinYiu/yolov7: Implementation of paper - YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors). Will this work for the transfer learning model? hailo_model_zoo/hailo_model_zoo/cfg/alls/generic/yolov7.alls at master · hailo-ai/hailo_model_zoo · GitHub

10

We are also open to retraining with a new object detector model or if you can link us to an example that runs a custom model and adds the postprocessing nodes after.

11

It would be helpful if you can link me to the model script mentioned here^

12

I have managed to convert my model and have a better understanding of the process after following this tutorial hailo-rpi5-examples/doc/retraining-example.md at main · hailo-ai/hailo-rpi5-examples · GitHub

It’s not clear from the DFC documentation that retraining requires what is discussed above