Onnx to Har parsing error

michele.tessari · November 20, 2024, 1:49pm

Hello,

I’m trying to export this onnx model in Har format using the following command:

hailo parser onnx models/superpoint_lightglue_pipeline_simpl.onnx\
 --net-name superpoint_lightglue_pipeline_simpl\
 --start-node-names images\
 --end-node-names keypoints matches mscores\
 --tensor-shapes images=[2,1,2560,2560]\
 --hw-arch hailo8\
 --input-format images=NCHW

but I get the following error:

Traceback (most recent call last):
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 235, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 316, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 367, in parse_model_to_hn
    fuser = HailoNNFuser(converter.convert_model(), net_name, converter.end_node_names)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py", line 83, in convert_model
    self._create_layers()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 40, in _create_layers
    self._add_direct_layers()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 122, in _add_direct_layers
    self._layer_callback_from_vertex(vertex)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 443, in _layer_callback_from_vertex
    self._create_ew_div_layer(vertex)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 1063, in _create_ew_div_layer
    self._create_ew_layer(vertex, EWDivLayer)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 1077, in _create_ew_layer
    const_layer, ew_op_input = self._create_ew_const_input_layer(vertex)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 1222, in _create_ew_const_input_layer
    const_layer = ConstInputLayer.create(f"{vertex.name}_input", [const_output_shape], input_values)
UnboundLocalError: local variable 'const_output_shape' referenced before assignment

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/local/workspace/hailo_virtualenv/bin/hailo", line 8, in <module>
    sys.exit(main())
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/tools/cmd_utils/main.py", line 111, in main
    ret_val = client_command_runner.run()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_platform/tools/hailocli/main.py", line 64, in run
    return self._run(argv)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_platform/tools/hailocli/main.py", line 104, in _run
    return args.func(args)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/tools/parser_cli.py", line 198, in run
    runner = self._parse(net_name, args, tensor_shapes)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/tools/parser_cli.py", line 282, in _parse
    runner.translate_onnx_model(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py", line 1158, in translate_onnx_model
    parser.translate_onnx_model(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 276, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 316, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 367, in parse_model_to_hn
    fuser = HailoNNFuser(converter.convert_model(), net_name, converter.end_node_names)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py", line 83, in convert_model
    self._create_layers()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 40, in _create_layers
    self._add_direct_layers()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 122, in _add_direct_layers
    self._layer_callback_from_vertex(vertex)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 443, in _layer_callback_from_vertex
    self._create_ew_div_layer(vertex)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 1063, in _create_ew_div_layer
    self._create_ew_layer(vertex, EWDivLayer)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 1077, in _create_ew_layer
    const_layer, ew_op_input = self._create_ew_const_input_layer(vertex)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 1222, in _create_ew_const_input_layer
    const_layer = ConstInputLayer.create(f"{vertex.name}_input", [const_output_shape], input_values)
UnboundLocalError: local variable 'const_output_shape' referenced before assignment

This is an already simplyfied version of the model using onnxsim, without optimization I get a different error:

Traceback (most recent call last):
  File "/local/workspace/hailo_virtualenv/bin/hailo", line 8, in <module>
    sys.exit(main())
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/tools/cmd_utils/main.py", line 111, in main
    ret_val = client_command_runner.run()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_platform/tools/hailocli/main.py", line 64, in run
    return self._run(argv)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_platform/tools/hailocli/main.py", line 104, in _run
    return args.func(args)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/tools/parser_cli.py", line 198, in run
    runner = self._parse(net_name, args, tensor_shapes)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/tools/parser_cli.py", line 282, in _parse
    runner.translate_onnx_model(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py", line 1158, in translate_onnx_model
    parser.translate_onnx_model(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 248, in translate_onnx_model
    raise e from None
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 235, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 316, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 367, in parse_model_to_hn
    fuser = HailoNNFuser(converter.convert_model(), net_name, converter.end_node_names)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py", line 83, in convert_model
    self._create_layers()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 39, in _create_layers
    self._update_vertices_info()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 217, in _update_vertices_info
    node.update_output_format()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 532, in update_output_format
    elif self.is_null_reshape()[0]:
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 5666, in is_null_reshape
    if self.is_reshape_before_split_windowed_attention() and len(self.get_output_shapes()) == 1:
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 5708, in is_reshape_before_split_windowed_attention
    input_shape[1] == output_shape[1] * output_shape[2]
IndexError: list index out of range

I’m running all inside the hailo_ai_sw_suite_2024-10 docker, so the Hailo DFC version is 3.29.0 and onnxsim is 0.4.36.

Thank you in advance

omria · November 24, 2024, 8:29am

Hey @michele.tessari

It seems like a combination of potential structural problems in your ONNX model and compatibility issues with the Hailo parser.

Let’s break down the issues and discuss the steps to resolve them:

Issue 1: const_output_shape UnboundLocalError

This error suggests that the Hailo parser is having trouble determining or assigning a shape to a constant tensor in your ONNX model. Here are a few things you can try:

Verify Tensor Shapes:
- Use ONNX tools to inspect your model and verify the tensor shapes of all intermediate layers.
- Run the command: onnxruntime-tools shape-infer <model.onnx> to infer and print the tensor shapes.
Simplify ONNX Model Further:
- Even though you’ve already simplified the model using onnxsim, it might still contain unsupported operations. Try applying additional simplification or pruning techniques to remove redundant nodes.
- Use the command: onnx-simplifier <model.onnx> --skip-fuse-bn to exclude operations like batch normalization if they are unnecessary.

Isolate Problematic Nodes:

Load the model in Python using ONNX tools and examine the node causing the issue:

import onnx
from onnx import shape_inference

model = onnx.load("model.onnx")
inferred_model = shape_inference.infer_shapes(model)

for node in inferred_model.graph.node:
    print(node.name, node.op_type)

Identify the node causing the const_output_shape error (likely a constant input).

Manual Adjustment:
- If you identify the problematic node, you may need to re-export or manually edit the ONNX file to ensure the constant has an explicit shape.

Issue 2: IndexError in is_reshape_before_split_windowed_attention

This error indicates a mismatch in input and output shapes for certain operations (likely Reshape). The Hailo parser expects all shapes to be explicitly defined and match architectural constraints. Here’s what you can try:

Check Reshape Nodes:
- Look for Reshape operations in your ONNX model.
- Verify that the input and output dimensions are compatible.
Inspect Specific Node:
- Identify the node causing the error using ONNX tools:
```
for node in model.graph.node:
    if node.op_type == "Reshape":
        print(node.name, node.attribute)
```
- Check whether the reshape operation depends on runtime tensors for defining dimensions (e.g., dynamic reshaping).

Fix Input Shape Mismatches:

For dynamic shapes, replace them with fixed ones.

Use ONNX Graph Surgeon:

from onnx_graphsurgeon import Graph

graph = Graph.from_onnx(model)
for node in graph.nodes:
    if node.op == "Reshape":
        # Modify shape inputs here
        node.inputs[1].values = [fixed_shape]

Save the modified ONNX file and retry the conversion.

I hope these steps help you resolve the issues you’re facing. If you have any further questions or need additional assistance, please don’t hesitate to reach out.

Best regards,
Omria

Topic		Replies	Views
Error while parsing .onnx file created from torch.nn.MultiHeadAttention General hailo8 , error	12	703	September 10, 2024
Parsing ONNX to HAR problem General hailo8	2	39	November 12, 2024
Correct Way To Export Torch Model to ONNX General compiler , onnxruntime , hailo8	2	90	October 31, 2024
Convert custom onnx model to har file General dfc , raspberry-pi , hailo8	2	177	September 27, 2024
Error on geting convert onnx model(yolov8) to hef format General	0	51	September 10, 2024

Onnx to Har parsing error

Related topics