Hi,
I am training yolo11n-seg with single label dust with the following command
yolo segment train
model=yolo11n-seg.pt
data=/path/to/dust.yaml
imgsz=640
epochs=150
batch=16
patience=30
seed=42
device=0
name=yolo11n_dust_640
*path: /path/to/dust
train: images/train
val: images/val
names:
0: dust
*
postprocessor used hailo-apps/hailo_apps/cpp/instance_segmentation/instance_seg_postprocess.cpp at 29acd11a8e641b2b61c3df45c95c97a775878e9f · hailo-ai/hailo-apps · GitHub
Its been observed quite a lot accuracy loss with INT8 quantization and hence precision_mode=a16_w16 is used with level=2 and compression=0 to improve the accuracy loss from the FP32 HAR/ONNX model bin. Note: level=3 and 4 are not supported with precision_mode=a16_w16.
Here is my model script yolov11n_dust_seg_calib.alls
normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])
change_output_activation(conv60, sigmoid)
change_output_activation(conv74, sigmoid)
change_output_activation(conv92, sigmoid)
quantization_param({conv*}, precision_mode=a16_w16)
quantization_param([dw1, dw2, dw3, dw4, dw5, dw6, dw7], precision_mode=a16_w16)
pre_quantization_optimization(
activation_clipping,
layers=[conv60, conv74, conv92],
mode=percentile,
clipping_values=[0.001, 99.999]
)
shortcut1 = shortcut(matmul1, [reduce_max_softmax1, ew_sub_softmax1])
allocator_param(automatic_reshapes=disabled)
model_optimization_flavor(optimization_level=2, compression_level=0, batch_size=1)
model_optimization_config(calibration, batch_size=8, calibset_size=7350)
Following two highlighted layers conv88 and conv89 for the 20x20 boxes, are coming as UINT8 in HEF. This seems to be a bug in the compiler as I have checked the same in FP32 HAR and Quantized HAR after loading and those layers seems to be ok as UINT16.
hailortcli parse-hef yolov11n_seg.hef
Architecture HEF was compiled for: HAILO8
Network group name: yolov11n_seg, Multi Context - Number of contexts: 4
Network name: yolov11n_seg/yolov11n_seg
VStream infos:
Input yolov11n_seg/input_layer1 UINT8, NHWC(640x640x3)
Output yolov11n_seg/conv88 UINT8, NHWC(20x20x64) ======> UINT8..why???
Output yolov11n_seg/conv92 UINT16, NHWC(20x20x1)
Output yolov11n_seg/conv89 UINT8, NHWC(20x20x32)=======>UINT8..why???
Output yolov11n_seg/conv70 UINT16, NHWC(40x40x64)
Output yolov11n_seg/conv74 UINT16, NHWC(40x40x1)
Output yolov11n_seg/conv71 UINT16, NHWC(40x40x32)
Output yolov11n_seg/conv54 UINT16, NHWC(80x80x64)
Output yolov11n_seg/conv60 UINT16, NHWC(80x80x1)
Output yolov11n_seg/conv55 UINT16, NHWC(80x80x32)
Output yolov11n_seg/conv57 UINT16, NHWC(160x160x32)
anyway in order to handle those two layers as UINT8 and rest as UNIT16, I made a patch in the postprocessor code -
*float dequantize_value(uint8_t val, float32_t qp_scale, float32_t qp_zp){
xt::xarray<uint16_t> get_xtensor_uint16_or_convert(HailoTensorPtr tensor)
{
if (tensor->vstream_info().format.type == HAILO_FORMAT_TYPE_UINT16) {
return common::get_xtensor_uint16(tensor);
}
if (tensor->vstream_info().format.type == HAILO_FORMAT_TYPE_UINT8) {
return xt::cast<uint16_t>(common::get_xtensor(tensor));
}
throw std::runtime_error("Unsupported output tensor datatype");
}*
But the result of all the box decoding is 0 as it seems HEF bin file got corrupted.
After a lot of trial and error, result is improved once those two conv layers and corresponding output_layer are explicitly made as a8_w8
*quantization_param([conv88, conv89, output_layer9, output_layer10],
precision_mode=a8_w8
)
*
Please let me know why compiler makes wrong precision_mode only for those two output layers?