I can succed to compile the my yolovv5m onnx to hef with nms, but when I test the hef with hailo8, the accuracy drops about 10%.
Then I test the compiling process in Emulator of SDK_QUANTIZED mode, it can detect the objects, but when I compile it to hef and run in C++ code, it still missed the objects with test image.
Here is the compiling and test code.
from hailo_sdk_client import ClientRunner
model_name = ‘i3yolov5m23’
onnx_path = f’…/models/{model_name}.onnx’
assert os.path.isfile(onnx_path), ‘Please provide valid path for ONNX file’
calib_dataset_new = np.load(‘i3calib_set_640_640.npy’)
test_dataset = np.load(‘i3test_im_set_640_640.npy’)
Initialize a new client runner
runner = ClientRunner(hw_arch=‘hailo8’)
Any other hw_arch can be used as well.
Translate YOLO model from ONNX
runner.translate_onnx_model(onnx_path, model_name,
start_node_names=[‘images’],
end_node_names=[‘Conv_344’, ‘Conv_309’, ‘Conv_274’],
net_input_shapes={‘images’: [1, 3, 640, 640]})
Note: NMS will be detected automatically, with a message that contains:
- ‘original layer name’: {‘w’: [WIDTHS], ‘h’: [HEIGHTS], ‘stride’: STRIDE, ‘encoded_layer’: TRANSLATED_LAYER_NAME}
Use nms_postprocess(meta_arch=yolov5) to add the NMS.
Add model script with NMS layer at the network’s output.
model_script_commands = [
‘normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])\n’,
‘resize_input1= resize(resize_shapes=[640,640])\n’,
‘nms_postprocess(meta_arch=yolov5, engine=cpu, nms_scores_th=0.2, nms_iou_th=0.4)\n’,
]
Note: Scores threshold of 0.0 means no filtering, 1.0 means maximal filtering. IoU thresholds are opposite: 1.0 means filtering boxes only if they are equal, and 0.0 means filtering with minimal overlap.
runner.load_model_script(‘’.join(model_script_commands))
Apply model script changes
runner.optimize_full_precision()
#runner.optimize(calib_dataset_new)
Infer an image with the Hailo Emulator
with runner.infer_context(InferenceContext.SDK_FP_OPTIMIZED) as ctx:
opt_nms_output = runner.infer(ctx, test_dataset[:16, …])
runner.optimize(calib_dataset_new)
# Infer an image with the Hailo Emulator
with runner.infer_context(InferenceContext.SDK_FP_OPTIMIZED) as ctx:
post_opt_nms_output = runner.infer(ctx, test_dataset[:16, …])
# Infer an image with the Hailo Emulator, can detect well in my test iamge
with runner.infer_context(InferenceContext.SDK_QUANTIZED) as ctx:
nms_output = runner.infer(ctx, test_dataset[:16, …])
Save the result state to a Quantized HAR file
quantized_model_har_path = f’{model_name}_quantized_model_nms.har’
runner.save_har(quantized_model_har_path)
hef = runner.compile()
file_name = f’{model_name}_nms.hef’
with open(file_name, ‘wb’) as f:
f.write(hef)