Error: While Parsing onnx Based models

I need to convert an open source onnx based model to hef based model , so for that i downloaded the DFC and ran the basic example and that was successful,but now when i do the same steps but with the other models that i downloaded from the onnx repo, so I get error I also tried with multiple different open source model , no one was able to be successfully parsed. The error atleast for this is



Now Guide me how do i make it ,compile successful , not only this i want to compile any of the opensource model,Guide me if I’m missing some technical details for the conversion.

Hey @avinash32.gahlowt ,

To convert an open-source ONNX model to a Hailo Executable Format (HEF) with the Dataflow Compiler (DFC), it’s essential to ensure compatibility with Hailo’s architecture and adhere to the compiler’s requirements. Here’s a step-by-step guide to address parsing issues and achieve a successful compilation:

Steps to Convert ONNX Models to HEF

1. Verify Model Compatibility

  • Not all ONNX models are directly compatible with Hailo’s architecture. Confirm that your model includes only supported layers (e.g., Convolution, ReLU, BatchNorm, Pooling) and excludes unsupported operations like dynamic reshaping or custom layers.
  • Consult Hailo’s documentation for a full list of supported layers and operations to ensure your model meets these requirements.

2. Preprocess the ONNX Model

  • Simplify the Model: Adjust the model to remove unsupported layers or complex operations. Tools like the ONNX Simplifier can reduce complexity by removing redundant nodes and standardizing operations.
  • Use Fixed Input Shapes: Hailo requires static dimensions for inputs, so replace any dynamic (wildcard) dimensions with specific values.

3. Set Input and Output Nodes Correctly

  • When using the translate_onnx_model function, set the start_node_names (input layers) and end_node_names (output layers) appropriately.
  • Ensure that the net_input_shapes dictionary matches the model’s expected input shape, as per your setup example.

4. Parse the Model with DFC

  • Specify Parameters: Make sure all necessary parameters are correctly specified in translate_onnx_model. Incorrect or missing node names can cause parsing issues.
  • Disable Shape Inference: If parsing fails due to shape inference issues, try setting disable_shape_inference=True in translate_onnx_model. Disabling shape inference can help avoid issues in complex models.

5. Troubleshoot Specific Errors

  • NoneType Error in Output Shapes: This error usually indicates that certain layers lack defined output shapes, often due to unsupported or complex operations. Run onnx.checker.check_model to validate that the ONNX model adheres to ONNX standards.
  • StopIteration Error: This error typically arises from incomplete parsing due to missing or incorrect node names. Double-check that start_node_names and end_node_names match nodes within the model.

6. Use Hailo SDK Tools for Debugging

  • Hailo’s SDK provides tools to inspect and validate model layers and shapes. Commands like hailo_sdk_client can verify that the model layers are correctly parsed and compatible with Hailo hardware.

Example Parsing Command

Here’s an example command based on the setup to convert an ONNX model:

runner = ClientRunner(hw_arch=chosen_hw_arch)
hn, npz = runner.translate_onnx_model(
    onnx_path,
    onnx_model_name,
    start_node_names=['input_node_name'],
    end_node_names=['output_node_name'],
    net_input_shapes={'input_node_name': [1, 3, 224, 224]},  # Modify as per model
    disable_shape_inference=True  # Use if shape inference causes issues
)

Common Adjustments

  • Dynamic Reshaping: Remove or modify any dynamic reshaping layers, as Hailo requires models to avoid dynamic reshaping for compatibility.
  • Input Shape Normalization: Ensure that input shapes are specified in NCHW format, which is Hailo’s standard input layout.
1 Like

Thanks @omria , I followed these steps

  1. I ran this code to print the output of the layers
    import onnx model = onnx.load('/home/avinash/Downloads/yolov4.onnx') for node in model.graph.node: print(node.op_type,end=" ")
    So it has few differences with the Hailo supported so i tried
`python3 -m onnxsim yolov4.onnx yolov4.onnx
Simplifying...
Finish! Here is the difference:
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃            ┃ Original Model ┃ Simplified Model ┃
┑━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
β”‚ Add        β”‚ 95             β”‚ 95               β”‚
β”‚ Cast       β”‚ 6              β”‚ 6                β”‚
β”‚ Concat     β”‚ 18             β”‚ 18               β”‚
β”‚ Constant   β”‚ 231            β”‚ 230              β”‚
β”‚ Conv       β”‚ 110            β”‚ 110              β”‚
β”‚ Exp        β”‚ 72             β”‚ 72               β”‚
β”‚ Gather     β”‚ 3              β”‚ 3                β”‚
β”‚ LeakyRelu  β”‚ 35             β”‚ 35               β”‚
β”‚ Log        β”‚ 72             β”‚ 72               β”‚
β”‚ MaxPool    β”‚ 3              β”‚ 3                β”‚
β”‚ Mul        β”‚ 72             β”‚ 72               β”‚
β”‚ Reshape    β”‚ 4              β”‚ 3                β”‚
β”‚ Resize     β”‚ 2              β”‚ 2                β”‚
β”‚ Shape      β”‚ 5              β”‚ 5                β”‚
β”‚ Sigmoid    β”‚ 6              β”‚ 6                β”‚
β”‚ Slice      β”‚ 8              β”‚ 8                β”‚
β”‚ Split      β”‚ 3              β”‚ 3                β”‚
β”‚ Tanh       β”‚ 72             β”‚ 72               β”‚
β”‚ Transpose  β”‚ 4              β”‚ 4                β”‚
β”‚ Model Size β”‚ 245.5MiB       β”‚ 245.6MiB         β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
`

So after this i checked the shape of model using this code

import onnx
model = onnx.load('yolov4.onnx')
input_names = [i.name for i in model.graph.input]
output_names = [o.name for o in model.graph.output]
print(f"Input names: {input_names}")
print(f"Output names: {output_names}")


# Get the shapes of the input tensors
input_shapes = {i.name: [dim.dim_value for dim in i.type.tensor_type.shape.dim] for i in model.graph.input}

# Print the input shapes
print("Input shapes:")
for input_name, shape in input_shapes.items():
    print(f"{input_name}: {shape}")

So After this i ran this to check if there is dynamic shaping if yes then change that

import onnx
model = onnx.load("yolov4.onnx")
for tensor in model.graph.input:
    if 'None' in str(tensor.type.tensor_type.shape.dim):
        tensor.type.tensor_type.shape.dim[0].dim_value = 1  # Batch size of 1
onnx.save(model, "yolov4.onnx")

Atfter that Shape convert

import onnx

# Load the ONNX model
model = onnx.load('yolov4.onnx')

# Update the shape of each input tensor (in this case, set the batch size to 1)
for input_tensor in model.graph.input:
    for dim in input_tensor.type.tensor_type.shape.dim:
        if dim.dim_value == 0:  # If batch size is dynamic (0)
            dim.dim_value = 1  # Set batch size to 1 (or another fixed number)

# Save the updated model
onnx.save(model, 'yolov4.onnx')

After executing all this the shape,start_node ,End Node are

python3 shape.py 
Input names: ['input_1:0']
Output names: ['Identity:0', 'Identity_1:0', 'Identity_2:0']
Input shapes:
input_1:0: [1, 416, 416, 3]

Before Shape Convert

python3 shape.py 
Input names: ['input_1:0']
Output names: ['Identity:0', 'Identity_1:0', 'Identity_2:0']
Input shapes:
input_1:0: [0, 416, 416, 3]

But After doing all this i’m still getting error like this


Also is there any way like you mentioned by Hailo sdk so is there any command hailo_sdk_client inspect or something by which one get to know whether the model is hailo compatible or not.
Because onnx Simplifier does not did a job.

Hi @avinash32.gahlowt,

We offer yolov4 in our ModelZoo. If you didn’t re-train it, you can download an already compiled model from there. Otherwise, I would recommend using our re-training docker.