Onnx to Har parsing error

General
,
1

Hello,

I’m trying to export this onnx model in Har format using the following command:

hailo parser onnx models/superpoint_lightglue_pipeline_simpl.onnx\
 --net-name superpoint_lightglue_pipeline_simpl\
 --start-node-names images\
 --end-node-names keypoints matches mscores\
 --tensor-shapes images=[2,1,2560,2560]\
 --hw-arch hailo8\
 --input-format images=NCHW

but I get the following error:

Traceback (most recent call last):
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 235, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 316, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 367, in parse_model_to_hn
    fuser = HailoNNFuser(converter.convert_model(), net_name, converter.end_node_names)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py", line 83, in convert_model
    self._create_layers()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 40, in _create_layers
    self._add_direct_layers()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 122, in _add_direct_layers
    self._layer_callback_from_vertex(vertex)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 443, in _layer_callback_from_vertex
    self._create_ew_div_layer(vertex)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 1063, in _create_ew_div_layer
    self._create_ew_layer(vertex, EWDivLayer)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 1077, in _create_ew_layer
    const_layer, ew_op_input = self._create_ew_const_input_layer(vertex)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 1222, in _create_ew_const_input_layer
    const_layer = ConstInputLayer.create(f"{vertex.name}_input", [const_output_shape], input_values)
UnboundLocalError: local variable 'const_output_shape' referenced before assignment

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/local/workspace/hailo_virtualenv/bin/hailo", line 8, in <module>
    sys.exit(main())
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/tools/cmd_utils/main.py", line 111, in main
    ret_val = client_command_runner.run()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_platform/tools/hailocli/main.py", line 64, in run
    return self._run(argv)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_platform/tools/hailocli/main.py", line 104, in _run
    return args.func(args)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/tools/parser_cli.py", line 198, in run
    runner = self._parse(net_name, args, tensor_shapes)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/tools/parser_cli.py", line 282, in _parse
    runner.translate_onnx_model(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py", line 1158, in translate_onnx_model
    parser.translate_onnx_model(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 276, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 316, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 367, in parse_model_to_hn
    fuser = HailoNNFuser(converter.convert_model(), net_name, converter.end_node_names)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py", line 83, in convert_model
    self._create_layers()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 40, in _create_layers
    self._add_direct_layers()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 122, in _add_direct_layers
    self._layer_callback_from_vertex(vertex)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 443, in _layer_callback_from_vertex
    self._create_ew_div_layer(vertex)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 1063, in _create_ew_div_layer
    self._create_ew_layer(vertex, EWDivLayer)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 1077, in _create_ew_layer
    const_layer, ew_op_input = self._create_ew_const_input_layer(vertex)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 1222, in _create_ew_const_input_layer
    const_layer = ConstInputLayer.create(f"{vertex.name}_input", [const_output_shape], input_values)
UnboundLocalError: local variable 'const_output_shape' referenced before assignment

This is an already simplyfied version of the model using onnxsim, without optimization I get a different error:

Traceback (most recent call last):
  File "/local/workspace/hailo_virtualenv/bin/hailo", line 8, in <module>
    sys.exit(main())
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/tools/cmd_utils/main.py", line 111, in main
    ret_val = client_command_runner.run()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_platform/tools/hailocli/main.py", line 64, in run
    return self._run(argv)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_platform/tools/hailocli/main.py", line 104, in _run
    return args.func(args)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/tools/parser_cli.py", line 198, in run
    runner = self._parse(net_name, args, tensor_shapes)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/tools/parser_cli.py", line 282, in _parse
    runner.translate_onnx_model(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_common/states/states.py", line 16, in wrapped_func
    return func(self, *args, **kwargs)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/runner/client_runner.py", line 1158, in translate_onnx_model
    parser.translate_onnx_model(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 248, in translate_onnx_model
    raise e from None
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 235, in translate_onnx_model
    parsing_results = self._parse_onnx_model_to_hn(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 316, in _parse_onnx_model_to_hn
    return self.parse_model_to_hn(
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/sdk_backend/parser/parser.py", line 367, in parse_model_to_hn
    fuser = HailoNNFuser(converter.convert_model(), net_name, converter.end_node_names)
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/translator.py", line 83, in convert_model
    self._create_layers()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/edge_nn_translator.py", line 39, in _create_layers
    self._update_vertices_info()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_translator.py", line 217, in _update_vertices_info
    node.update_output_format()
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 532, in update_output_format
    elif self.is_null_reshape()[0]:
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 5666, in is_null_reshape
    if self.is_reshape_before_split_windowed_attention() and len(self.get_output_shapes()) == 1:
  File "/local/workspace/hailo_virtualenv/lib/python3.10/site-packages/hailo_sdk_client/model_translator/onnx_translator/onnx_graph.py", line 5708, in is_reshape_before_split_windowed_attention
    input_shape[1] == output_shape[1] * output_shape[2]
IndexError: list index out of range

I’m running all inside the hailo_ai_sw_suite_2024-10 docker, so the Hailo DFC version is 3.29.0 and onnxsim is 0.4.36.

Thank you in advance

2

Hey @michele.tessari

It seems like a combination of potential structural problems in your ONNX model and compatibility issues with the Hailo parser.

Let’s break down the issues and discuss the steps to resolve them:

Issue 1: const_output_shape UnboundLocalError

This error suggests that the Hailo parser is having trouble determining or assigning a shape to a constant tensor in your ONNX model. Here are a few things you can try:

  1. Verify Tensor Shapes:

    • Use ONNX tools to inspect your model and verify the tensor shapes of all intermediate layers.
    • Run the command: onnxruntime-tools shape-infer <model.onnx> to infer and print the tensor shapes.

  2. Simplify ONNX Model Further:

    • Even though you’ve already simplified the model using onnxsim, it might still contain unsupported operations. Try applying additional simplification or pruning techniques to remove redundant nodes.
    • Use the command: onnx-simplifier <model.onnx> --skip-fuse-bn to exclude operations like batch normalization if they are unnecessary.

  3. Isolate Problematic Nodes:

    • Load the model in Python using ONNX tools and examine the node causing the issue:

      import onnx
from onnx import shape_inference

model = onnx.load("model.onnx")
inferred_model = shape_inference.infer_shapes(model)

for node in inferred_model.graph.node:
    print(node.name, node.op_type)

    • Identify the node causing the const_output_shape error (likely a constant input).

  4. Manual Adjustment:

    • If you identify the problematic node, you may need to re-export or manually edit the ONNX file to ensure the constant has an explicit shape.

Issue 2: IndexError in is_reshape_before_split_windowed_attention

This error indicates a mismatch in input and output shapes for certain operations (likely Reshape). The Hailo parser expects all shapes to be explicitly defined and match architectural constraints. Here’s what you can try:

  1. Check Reshape Nodes:

    • Look for Reshape operations in your ONNX model.
    • Verify that the input and output dimensions are compatible.

  2. Inspect Specific Node:

    • Identify the node causing the error using ONNX tools:

      for node in model.graph.node:
    if node.op_type == "Reshape":
        print(node.name, node.attribute)

    • Check whether the reshape operation depends on runtime tensors for defining dimensions (e.g., dynamic reshaping).

  3. Fix Input Shape Mismatches:

    • For dynamic shapes, replace them with fixed ones.

    • Use ONNX Graph Surgeon:

      from onnx_graphsurgeon import Graph

graph = Graph.from_onnx(model)
for node in graph.nodes:
    if node.op == "Reshape":
        # Modify shape inputs here
        node.inputs[1].values = [fixed_shape]

    • Save the modified ONNX file and retry the conversion.

I hope these steps help you resolve the issues you’re facing. If you have any further questions or need additional assistance, please don’t hesitate to reach out.

Best regards,
Omria