Custom Yolov11 not detecting after compilation

General
,
1

Hello,
I’ve compiled my ONNX file using the Hailo Dataflow Compiler. The ONNX file was retrained with custom data based on YOLOv11s. After testing the model on my Hailo, the performance is very poor — it can’t detect most of the objects. Note that I tested the original .pt model, and it performs well. I also previously compiled a custom YOLOv8 model, and it worked fine. Additionally, I have already checked the output node names for YOLOv11, and the compilation didn’t raise any errors.

Here is the .alls file

normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])
change_output_activation(conv54, sigmoid)
change_output_activation(conv65, sigmoid)
change_output_activation(conv80, sigmoid)
model_optimization_flavor(optimization_level=4)
nms_postprocess("nms_layer_config.json", meta_arch=yolov8, engine=cpu)

and the .json:

{
	"nms_scores_th": 0.3,
	"nms_iou_th": 0.7,
	"image_dims": [
		640,
		640
	],
	"max_proposals_per_class": 100,
	"classes": 1,
	"regression_length": 16,
	"background_removal": false,
	"background_removal_index": 0,
	"bbox_decoders": [
		{
			"name": "bbox_decoder51",
			"stride": 8,
			"reg_layer": "conv51",
			"cls_layer": "conv54"
		},
		{
			"name": "bbox_decoder62",
			"stride": 16,
			"reg_layer": "conv62",
			"cls_layer": "conv65"
		},
		{
			"name": "bbox_decoder77",
			"stride": 32,
			"reg_layer": "conv77",
			"cls_layer": "conv80"
		}
	]
}

I also lowered the optimization level to 2, but nothing changed.

Has anyone else had issues with YOLOv11? I’m very confused. I don’t know what could be wrong. I followed the instructions from this guide: Guide to using the DFC to convert a modified YoloV11 on Google Colab and also recompiled using the CLI commands instead.

Thanks!