Issues detecting all objects with Hailo 8L, RPi5, yolov8n_seg, HailoRT 4.20

General
1

Hi,

i’ve trained a yolov8 segmentation model with 3000 images. These images contain several thousand eggs - only 1 class. Segmenting these eggs via YoloV8 running on CUDA works perfectly (<0,1% of all eggs are not detected).

On Hailo8L some eggs are not detected at all and confidence values are relatively low on some (10% of all eggs are not detected).

I have converted the best.pt into onnx and compiled it with hailomz

yolo export model=./best.pt imgsz=640 format=onnx opset=11
hailomz compile yolov8n_seg --ckpt ./best.onnx --hw-arch hailo8l --calib-path ./dataset-yolov8/ --performance

My yolov8n_seg.alls is modifed (according to some suggestions from the community for low class count models):

normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])
change_output_activation(conv45, sigmoid)
change_output_activation(conv61, sigmoid)
change_output_activation(conv74, sigmoid)
model_optimization_flavor(optimization_level=1, compression_level=0, batch_size=2)
post_quantization_optimization(adaround, policy=enabled)
performance_param(compiler_optimization_level=max)

I also tried quantization_param([conv45, conv61, conv74], force_range_out=[0.0, 1.0]).

Output of hailomz is:

<Hailo Model Zoo INFO> Start run for network yolov8n_seg ...
<Hailo Model Zoo INFO> Initializing the hailo8l runner...
[info] Translation started on ONNX model yolov8n_seg
[info] Restored ONNX model yolov8n_seg (completion time: 00:00:00.06)
[info] Extracted ONNXRuntime meta-data for Hailo model (completion time: 00:00:00.30)
[info] Start nodes mapped from original model: 'images': 'yolov8n_seg/input_layer1'.
[info] End nodes mapped from original model: '/model.22/cv2.2/cv2.2.2/Conv', '/model.22/cv3.2/cv3.2.2/Conv', '/model.22/cv4.2/cv4.2.2/Conv', '/model.22/cv2.1/cv2.1.2/Conv', '/model.22/cv3.1/cv3.1.2/Conv', '/model.22/cv4.1/cv4.1.2/Conv', '/model.22/cv2.0/cv2.0.2/Conv', '/model.22/cv3.0/cv3.0.2/Conv', '/model.22/cv4.0/cv4.0.2/Conv', '/model.22/proto/cv3/act/Mul'.
[info] Translation completed on ONNX model yolov8n_seg (completion time: 00:00:00.97)
[info] Saved HAR to: /local/shared_with_docker/yolov8n_seg.har
<Hailo Model Zoo INFO> Using generic alls script found in /local/workspace/hailo_model_zoo/hailo_model_zoo/cfg/alls/generic/yolov8n_seg.alls because there is no specific hardware alls
<Hailo Model Zoo INFO> Preparing calibration data...
[info] Loading model script commands to yolov8n_seg from /local/workspace/hailo_model_zoo/hailo_model_zoo/cfg/alls/generic/yolov8n_seg.alls
[info] ParsedPerformanceParam command, setting optimization_level(max=2)
[info] Starting Model Optimization
[info] Model received quantization params from the hn
[info] MatmulDecompose skipped
[info] Starting Mixed Precision
[info] Model Optimization Algorithm Mixed Precision is done (completion time is 00:00:00.70)
[info] LayerNorm Decomposition skipped
[info] Starting Statistics Collector
[info] Using dataset with 64 entries for calibration
Calibration: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 64/64 [00:33<00:00,  1.94entries/s]
[info] Model Optimization Algorithm Statistics Collector is done (completion time is 00:00:35.68)
[info] Output layer yolov8n_seg/conv45 with sigmoid activation was detected. Forcing its output range to be [0, 1] (original range was [1.0547330942448013e-22, 0.28611060976982117]).
[info] Output layer yolov8n_seg/conv61 with sigmoid activation was detected. Forcing its output range to be [0, 1] (original range was [0.0, 0.9814704060554504]).
[info] Output layer yolov8n_seg/conv74 with sigmoid activation was detected. Forcing its output range to be [0, 1] (original range was [5.5641644070014706e-11, 0.9697117805480957]).
[info] Starting Fix zp_comp Encoding
[info] Model Optimization Algorithm Fix zp_comp Encoding is done (completion time is 00:00:00.00)
[info] Matmul Equalization skipped
[info] Finetune encoding skipped
[info] Starting Bias Correction
[info] The algorithm Bias Correction will use up to 1.15 GB of storage space
[info] Using dataset with 64 entries for Bias Correction
Bias Correction: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 84/84 [04:04<00:00,  2.91s/blocks, Layers=['yolov8n_seg/conv48_output_0']]
[info] Model Optimization Algorithm Bias Correction is done (completion time is 00:04:06.41)
[info] Adaround skipped
[info] Quantization-Aware Fine-Tuning skipped
[info] Starting Layer Noise Analysis
Full Quant Analysis: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [02:56<00:00, 22.12s/iterations]
[info] Model Optimization Algorithm Layer Noise Analysis is done (completion time is 00:03:00.87)
[info] Output layers signal-to-noise ratio (SNR): measures the quantization noise (higher is better)
[info]  yolov8n_seg/output_layer9 SNR:  17.52 dB
[info]  yolov8n_seg/output_layer10 SNR: 25.28 dB
[info]  yolov8n_seg/output_layer8 SNR:  16.15 dB
[info]  yolov8n_seg/output_layer5 SNR:  16.22 dB
[info]  yolov8n_seg/output_layer6 SNR:  19.32 dB
[info]  yolov8n_seg/output_layer7 SNR:  12.29 dB
[info]  yolov8n_seg/output_layer1 SNR:  20.37 dB
[info]  yolov8n_seg/output_layer2 SNR:  0.0 dB
[info]  yolov8n_seg/output_layer3 SNR:  16.74 dB
[info]  yolov8n_seg/output_layer4 SNR:  15.92 dB
[info] Model Optimization is done
[info] Saved HAR to: /local/shared_with_docker/yolov8n_seg.har
<Hailo Model Zoo INFO> Using generic alls script found in /local/workspace/hailo_model_zoo/hailo_model_zoo/cfg/alls/generic/yolov8n_seg.alls because there is no specific hardware alls
[info] Loading model script commands to yolov8n_seg from /local/workspace/hailo_model_zoo/hailo_model_zoo/cfg/alls/generic/yolov8n_seg.alls
[info] ParsedPerformanceParam command, setting optimization_level(max=2)
[info] ParsedPerformanceParam command, setting optimization_level(max=2)
[info] Appending model script commands to yolov8n_seg from string
[info] ParsedPerformanceParam command, setting optimization_level(max=2)
[info] Loading network parameters
[info] Starting Hailo allocation and compilation flow
[info] Finding the best partition to contexts...
[.........................<==>...........] Duration: 00:00:32                                                                                                                                                                                                      
Iteration Done
...

Would an NMS inside the alls file help optimization? I tried
nms_postprocess(meta_arch=yolov8, engine=cpu), but it throws an error, as my model is _seg and a yolov8_seg is not available for nms_postprocess. The output_layer2 SNR is 0 dB, that’s ok? Any other ideas?

Regards

…Alex

2

Hi @alexbrickwedde,

Hailo still doesn’t support hailort_postprocess for yolov8_seg.

The fact that output_layer2 has SNR 0 dB is not good at all.

You did a good job with the compilation flow. The only things that would be good to fix is not mixing IBC and adaround, IBC is already part of the adaround algorithm. This means changing:

model_optimization_flavor(optimization_level=1, compression_level=0, batch_size=2)

to:

model_optimization_flavor(optimization_level=3, compression_level=0, batch_size=2)

Also, the default number of epochs for adaround is 320, which can cause over-fitting. I would do something like:

post_quantization_optimization(adaround, policy=enabled, epochs=5)

How do you postprocess/visualize the results?

3

Thank you, I’ll try that.

I implemented NMS, but also let it draw all rectangles - just to be sure NMS is not the cause. In the moment i don’t use the segmentation results, since calculation on the pi takes too much time and lowers FPS (I could switch to simple detection…)

Could it also be, that the dataset format for optimization is not correct? IIRC i’ll have to feed the “original” model with BGR images for prediction. I tried both RGB and BGR with Hailo prediction, but not for optimization.

4

The calibset could be the issue.

  • If RGB/BGR depends on your training, whatever you used in the training should be also used in the optimization and inference
  • What’s the original resolution of the images? Extreme resizing could cause artifacts
  • How many images are you using for the adaround?
5

With an BGR calibset no egg is detected at all, so that’s probably not the issue.

The images (training and inference) are 640x640. Many of the trained images were recorded on the same device where the inference runs.

Optimization runs with roughly 1600 images. What does “Using dataset with 64 entries for bias correction” mean?