Model parsing error (ReduceMean, Unsqueeze)

General
, ,
1

Hi everyone,

I am currently trying to run inference on a custom model with Hailo8. When I attempt to parse the model’s ONNX file using Hailo’s Data Flow Compiler, I encounter the following unsupported layer error.the version of the Data Flow Compiler is 2024_04.

UnexpectedNodeError in op /Unsqueeze_1: Unexpected node /Unsqueeze_1 (Unsqueeze)
UnsupportedReduceMeanLayerError in op /_attention/layer_norm/ReduceMean: Reduce mean layer /_attention/layer_norm/ReduceMean has unsupported axis -1 (must be over one spatial dimension only).
UnsupportedReduceMeanLayerError in op /_mlp/layer_norm/ReduceMean: Reduce mean layer /_mlp/layer_norm/ReduceMean has unsupported axis 2 (must be over one spatial dimension only).
UnsupportedReduceMeanLayerError in op /_attention/layer_norm/ReduceMean_1: Reduce mean layer /_attention/layer_norm/ReduceMean_1 has unsupported axis 2 (must be over one spatial dimension only).
UnsupportedReduceMeanLayerError in op /_mlp/layer_norm/ReduceMean_1: Reduce mean layer /_mlp/layer_norm/ReduceMean_1 has unsupported axis 2 (must be over one spatial dimension only).
UnsupportedFeatureSplitterLayerError in op /_attention/Split: Feature splitter vertex /_attention/Split is splitting input over unsupported axis 2
UnexpectedNodeError in op /Unsqueeze: Unexpected node /Unsqueeze (Unsqueeze)

However, according to the Data Flow Compiler user guide, both the ReduceMean and Unsqueezed layers are supposed to be supported.

Why am I getting the unsupported layer error?

2

It will be hard getting to the bottom of just feom that. Maybe it will be more efficient to open a support ticket, with attached onnx.

3

unsuqeezed layer can used only in specific cases such as
between Dense and Conv layers.

The manual says that Average Pooling(Reduce Mean) works fine, but sometimes it doesn’t work for unknown reasons.

I suggest some solutions.

First, update DFC and HRT newest verstion
Second, Translate from TFLite or TF to hef instead of onnx.
Last, Converts unsupported layers from hailo to supported layers.

Thanks.

4

Are you using openpilot’s supercombo.onnx model? I got the same errors.

1 Like
5

Hi,
From some point in the model, the dimensions are reduced to 3, which is a problem:
image
Been able to parse (after simplifier) up untli there:
hailo parser onnx supercombo-sim.onnx --end-node-names Conv_200 --start-node-names Conv_0

6

Yes, I am using the supercombo.onnx model.
As you pointed out, I was able to compile the model and perform inference up to a certain point. However, if possible, I would prefer to run the entire supercombo.onnx model, including the final layers, on Hailo8.

1 Like
7

@Nadav

Hi,

I was able to specify an intermediate point in the ML model using --end-node-names and perform inference up to that point with Hailo. For the remaining part, I currently use a tool I have to edit the ONNX and create an ONNX file that performs inference only from the point after --end-node-names.
However, I believe that the Hailo dataflow compiler tool might have a feature to offload the remaining part to the CPU. When performing inference with Hailo up to an intermediate point in the ML model, how is the remaining part expected to be inferred?

8

Hi @koki.igari,
The remaining part of the ONNX graph is stored in the compiled HAR, you can view this using the hailor har info command:
image

You can extract that from the HAR using:
hailo har extract supercombo-sim.har --postprocess-model-path post_proc.onnx

Now you have the ONNX, and you can execute it using the onnxruntime package.

1 Like
9

Indeed, with that procedure, I was able to obtain the remaining parts of the ONNX file. Thank you.