The results of simulator are different from onnx

[FrozenStream]

My task is to translate a onnx model to hef.

At the beginning, the input of onnx is dynamic like['batch', 3, 'ax1', 'ax2']

I test this model it using onnxruntime on Python3:

ort_sess = onnxruntime.InferenceSession("./ppocr_det.onnx")
print(ort_sess.get_inputs()[0])

ort_inputs = {ort_sess.get_inputs()[0].name: x}
ort_outs = ort_sess.run(None, ort_inputs)

and it returned correctly.
Everything went well till now when I was trying to convert it to har file and test it in Emulator

from hailo_sdk_client import ClientRunner, InferenceContext
chosen_hw_arch = "hailo8l"

model_name = "ppocr_v4"
onnx_path = "ppocr_det.onnx"
hailo_model_har = f"{model_name}_hailo_model.har"

# translate onnx to har
runner = ClientRunner(hw_arch=chosen_hw_arch)
hn, npz = runner.translate_onnx_model(
    onnx_path,
    model_name,
    start_node_names='x',
    end_node_names='sigmoid_0.tmp_0',
    net_input_shapes={"x": [1, 3, 640, 640]},
)
runner.save_har(hailo_model_har)

import cv2
import numpy as np
import matplotlib.pyplot as plt

# test native model
runner = ClientRunner(har=hailo_model_har)

images_path = "../data"
img = cv2.imread('IMG_0599.jpg', cv2.IMREAD_COLOR)
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img = img / 255.
img = img.astype(np.float32)
img=cv2.resize(img,(640,640))
img = np.array([img])

with runner.infer_context(InferenceContext.SDK_NATIVE) as ctx:
    native_res = runner.infer(ctx, img)

At this point, a completely incorrect return value was obtained.

My question is:

1. The input dimension of onnx is [1, 3, 640, 640], while the input dimension of the har file I obtained is[-1, 640, 640, 3]. Is this correct? What do these dimensions refer to respectively?
2. I tried to use the same image as Input of onnx and har. For the har file, I tried to skip the transport step and keep the [h, w, c] format as input. Unfortunately, I got completely different return, how can i fix it?

[omria]

Hey @FrozenStream ,

Question 1

Yes, these dimensions are correct but they’re in different formats. Let me break down what each dimension means:

ONNX format [1, 3, 640, 640] (NCHW):

• 1: Batch size (fixed at 1)
• 3: Number of channels (RGB)
• 640, 640: Image height and width

HAR format [-1, 640, 640, 3] (NHWC):

• -1: Dynamic batch size (can handle variable batch sizes)
• 640, 640: Image height and width
• 3: Number of channels (RGB)

Question 2

You’re getting different results because the HAR file expects input in NHWC format, while you’re providing it with NCHW format. Here’s how to fix it:

1. Add the transpose step before HAR inference:

# Preprocess image
img = cv2.imread('your_image.jpg', cv2.IMREAD_COLOR)
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img = img / 255.0  # Normalize
img = img.astype(np.float32)
img = cv2.resize(img, (640, 640))
img = np.array([img])  # Add batch dimension

# Critical step: Convert to NHWC for HAR inference
img = np.transpose(img, (0, 2, 3, 1))  # From NCHW to NHWC

# HAR inference
with runner.infer_context(InferenceContext.SDK_NATIVE) as ctx:
    native_res = runner.infer(ctx, img)

Let me know if you need help with any of these steps or if you’re still seeing issues!

[FrozenStream]

Emm… In fact, that’s how I used it, otherwise it wouldn’t work at all.
the input image like:

I got from onnx:

and I got from emulator:

import cv2
import numpy as np
import matplotlib.pyplot as plt

runner = ClientRunner(har=hailo_model_har)

img = cv2.imread('IMG_0599.jpg', cv2.IMREAD_COLOR)
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img = cv2.resize(img,(640,640)) / 255.
img = img.astype(np.float32)
img = np.array([img])
print(img.shape)

plt.imshow(img[0])
plt.show()

with runner.infer_context(InferenceContext.SDK_NATIVE) as ctx:
    native_res = runner.infer(ctx, img)

and that is my onnx model,

[FrozenStream]

Hey guys, after using onnxsim to process the onnx model, I successfully solved this problem. But why?