Correct Way To Export Torch Model to ONNX

General
, ,
1

Hello Hailo Community,

As many of you, I want to run my own models on the Hailo8L.
To achieve this I am converting my Torch models to ONNX using:

torch.onnx.export(
        model.eval().cpu(),
        input,
        'model.onnx',
        export_params=True,
        do_constant_folding=True,
        input_names=['input'],
        output_names=['a', 'b', 'c'],
        opset_version=17,
        dynamic_axes={
            'input': {0: 'batch_size'},
            'a': {0: 'batch_size'},
            'b': {0: 'batch_size'},
            'c': {0: 'batch_size'},
        },
    )

However, when running

hailo parser onnx --hw-arch hailo8l  model.onnx

I get the following error

 Parsing failed with recommendations for end node names: ['_v_382'].

Even when specifying the end node names the error persists.

When examining the ONNX model in Netron the correct a/b/c end nodes are shown.

Also, the ONNX model behaves correctly when using OpenVino for inference.

How could I correctly export a Torch model to ONNX and convert it to HAR and finally to HEF for inference on the Hailo8L?

2

Hey @m.wijkhuizen

Welcome to the Hailo Community!

To better assist you, could you share some details about your ONNX model? Specifically, we’d need to verify:

  1. Model Configuration
  • What ONNX opset version are you using?
  • How are your dynamic axes defined?
  • Which ONNX version did you export with?
  1. Export Settings
  • Could you confirm your export parameters?
  • How did you handle the batch size configuration?

These details will help us identify if there’s a mismatch between your model settings and Hailo’s requirements.

Looking forward to helping you resolve this!

3
  • ONNS Opset 17 is used, the latest supported by Hailo.
  • Dynamic axes are defined as in the question post.

The problem seems to be that Torch is channel first, thus NxCxHxW, but the HAR format is channel last NxHxWxC.

When performing operations on a static dimension, this is not correctly translated to the HAR format.

Additionally, with a single channel, the squeeze operation on dimension 1 works perfectly fine in Torch and ONNX, but is giving errors when converting to HAR.

The whole channel first/last seems to be a source of errors and I haven’t figured out how to fix this.