Wrong dimensions in har file from onnx containing torch.nn.ConvTranspose2d layers

General
, ,
1

Hello, I have a problem when converting an onnx file that contains ConvTranspose2d layers to har: the output dimensions of the 2 files do not match.

For example, given an input of dimension (3,24,14) that goes through a nn.ConvTranspose2d(in_channels=3 , out_channels=9 , kernel_size=3), the output (in pytorch and onnx) has dimension (9,26,16) while the har model output has dimension (9,24,14).

The har model is generated by

hailo parser onnx --hw-arch hailo8l model.onnx
2

Hey @sorbafabrizio,

The output dimension mismatch between your ONNX and HAR models for the ConvTranspose2d layer likely stems from differences in how padding and stride parameters are interpreted during conversion. Here are some steps to troubleshoot:

  1. Verify padding, stride, and dilation parameters are correctly translated from PyTorch to ONNX to HAR.

  2. Use Netron to check if ONNX intermediate layer dimensions match the PyTorch model before HAR conversion.

  3. Review Hailo parser documentation for any specific handling of ConvTranspose2d layers or hardware compatibility requirements.

  4. Consider adjusting the model architecture (e.g., modifying padding or kernel sizes) to ensure consistent output dimensions post-conversion.

  5. If the issue persists, consult Hailo support or documentation for known issues with ConvTranspose2d layer conversion.

Let me know if you need any clarification or further assistance!

Best Regards,
Omri

3

Thanks @omria,

yes from documentation it appears that only SAME_TENSORFLOW padding is supported.
After fixing that in the ConvTranspose2d layer, the output dimensions of the har and onnx match.

4

@omria,

I still have troubles getting the hef output right.

I am testing this simple network (trained to return the input)

class Autoencoder(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = nn.Conv2d(in_channels=3,out_channels=3,kernel_size=3, stride=1, padding=1)
        
    def forward(self, x):
        x = self.conv(x)
        return x

After conversion, the dimensions and weights in the har file match those in the onnx file. However, when I run the hef model, the output does not match the one from the onnx model.
Here are the outputs of the onnx and hef for a test input image:

Onnx output
out onnx

Hef output
out hailo

Do you have any idea what the problem could be?
I tried to rearrange the tensor dimensions in different ways, but I always get the same kind of output.

Further info:

The onnx file is generated by

model = Autoencoder()
model.load_state_dict(torch.load("model.tar"))
dummy_in = torch.randn(1,3,384,216)
torch.onnx.export(model, dummy_in, 'model.onnx', export_params=True, opset_version=15)

and the hef file by

hailo parser onnx --hw-arch hailo8l model.onnx
hailo optimize --hw-arch hailo8l --use-random-calib-set model.har 
hailo compiler --hw-arch hailo8l model_optimized.har

The hef is run on a raspberry pi by

import cv2
from picamera2.devices import Hailo

hailo = Hailo("model.hef")
img_test=cv2.imread("test.png")
out = hailo.run(img_test)
5

@omnia,

nevermind, I solved the issue by changing the input shape with the --tensor-shapes parameter in the hailo parser:

hailo parser onnx --hw-arch hailo8l model_fp32.onnx --tensor-shapes [1,3,216,384]
1 Like