Can detect objects well with Emulator in SDK_QUANTIZED mode, but missed some objects when compile and run with hef in hailo8

I can succed to compile the my yolovv5m onnx to hef with nms, but when I test the hef with hailo8, the accuracy drops about 10%.

Then I test the compiling process in Emulator of SDK_QUANTIZED mode, it can detect the objects, but when I compile it to hef and run in C++ code, it still missed the objects with test image.

Here is the compiling and test code.

from hailo_sdk_client import ClientRunner

model_name = ‘i3yolov5m23’
onnx_path = f’…/models/{model_name}.onnx’
assert os.path.isfile(onnx_path), ‘Please provide valid path for ONNX file’

calib_dataset_new = np.load(‘i3calib_set_640_640.npy’)
test_dataset = np.load(‘i3test_im_set_640_640.npy’)

Initialize a new client runner

runner = ClientRunner(hw_arch=‘hailo8’)

Any other hw_arch can be used as well.

Translate YOLO model from ONNX

runner.translate_onnx_model(onnx_path, model_name,
start_node_names=[‘images’],
end_node_names=[‘Conv_344’, ‘Conv_309’, ‘Conv_274’],
net_input_shapes={‘images’: [1, 3, 640, 640]})

Note: NMS will be detected automatically, with a message that contains:

- ‘original layer name’: {‘w’: [WIDTHS], ‘h’: [HEIGHTS], ‘stride’: STRIDE, ‘encoded_layer’: TRANSLATED_LAYER_NAME}

Use nms_postprocess(meta_arch=yolov5) to add the NMS.

Add model script with NMS layer at the network’s output.

model_script_commands = [
‘normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])\n’,
‘resize_input1= resize(resize_shapes=[640,640])\n’,
‘nms_postprocess(meta_arch=yolov5, engine=cpu, nms_scores_th=0.2, nms_iou_th=0.4)\n’,
]

Note: Scores threshold of 0.0 means no filtering, 1.0 means maximal filtering. IoU thresholds are opposite: 1.0 means filtering boxes only if they are equal, and 0.0 means filtering with minimal overlap.

runner.load_model_script(‘’.join(model_script_commands))

Apply model script changes

runner.optimize_full_precision()

#runner.optimize(calib_dataset_new)

Infer an image with the Hailo Emulator

with runner.infer_context(InferenceContext.SDK_FP_OPTIMIZED) as ctx:
opt_nms_output = runner.infer(ctx, test_dataset[:16, …])

runner.optimize(calib_dataset_new)
# Infer an image with the Hailo Emulator
with runner.infer_context(InferenceContext.SDK_FP_OPTIMIZED) as ctx:
post_opt_nms_output = runner.infer(ctx, test_dataset[:16, …])

# Infer an image with the Hailo Emulator, can detect well in my test iamge

with runner.infer_context(InferenceContext.SDK_QUANTIZED) as ctx:
nms_output = runner.infer(ctx, test_dataset[:16, …])

Save the result state to a Quantized HAR file

quantized_model_har_path = f’{model_name}_quantized_model_nms.har’
runner.save_har(quantized_model_har_path)

hef = runner.compile()

file_name = f’{model_name}_nms.hef’
with open(file_name, ‘wb’) as f:
f.write(hef)

Can you show example picture diff between emulator and HW?