Change precision mode of model.hef - Yolov8

General
1

I have trained Yolov8 using the given docker, and i successfully converted it into HEF model.
Now my question is what is the default precision of the converted model i.e. HEF Model. AS per my exploration i got to know it is UINT8 but i am not sure about that.

Also my requirement is to create model with following precision modes:

  • Float32
  • UINT16

In Conversion, I have seen some logs regarding this but unable to get the points like where are these params. what is mixed precision etc

[info] Model received quantization params from the hn
[info] Starting Mixed Precision
[info] Mixed Precision is done (completion time is 00:00:00.06)
2

Hi @tahseenabbas,
The default precision is 8b for weights, and 8b for activation (tensors), 16b for bias.

The DFC has the ability to change the numeric representation of the weights to 4b or 16b, and the activation to 16b.

All of these are UINT. You can direct the compiler to use 16b representation on all the layers taht support that. but it will have an advert effect on the throughput, therefore we usually recommend using the 16b only on very specific layers, in case they induce large noise.

Float32 is currently not supported in our product.

3

Thanks for your time,
but i was wondering is there any other way to change 8b precision to 16b except changing the model.alls file ?
Because right now i am not that much sure how to select the layers which are suitable to be converted to 16b without affecting the accuracy that much.

4

Technically, there is no other way to change the numeric representation but using the alls file in the optimization phase.

Finding the suitable the layers can be more tricky. The best way to identify which layers are most suitable for 16b, is to use the report from the layer analysis tool

5

Alright, Thank you so much
I’ll try and get back to you on this

6

I have added these layers as 16b

normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])
change_output_activation(conv42, sigmoid)
change_output_activation(conv53, sigmoid)
change_output_activation(conv63, sigmoid)
quantization_param(conv2, precision_mode=a16_w16)
quantization_param(conv4, precision_mode=a16_w16)
quantization_param(conv5, precision_mode=a16_w16)
quantization_param(conv6, precision_mode=a16_w16)
quantization_param(conv35, precision_mode=a16_w16)
quantization_param(conv37, precision_mode=a16_w16)
quantization_param(conv38, precision_mode=a16_w16)
quantization_param(conv46, precision_mode=a16_w16)
quantization_param(conv57, precision_mode=a16_w16)
quantization_param(conv58, precision_mode=a16_w16)
quantization_param(conv62, precision_mode=a16_w16)
quantization_param(conv63, precision_mode=a16_w16)
nms_postprocess("../../postprocess_config/yolov8s_nms_config.json", meta_arch=yolov8, engine=cpu)

and getting this error

[error] Mapping Failed (allocation time: 1m 18s)
Value 80,008 is not possible to represent in 2s complement using 13 bits. Please check quantization values for node: conv41.

[error] Failed to produce compiled graph
[error] BackendAllocatorException: Compilation failed: Value 80,008 is not possible to represent in 2s complement using 13 bits. Please check quantization values for node: conv41.

I have tried removing some layers but this error keeps popping with any other layer number? Is there anything i am not doing in a correct way ?

7

Are you using that alls before the optimize step?

8

No, i am using it while compilation.

10

After training yolov8 model, i exported it to onnx and changed that yolov8s.alls file as i have already sent. and then i am running following command to compile it

hailomz compile --ckpt weights/best.onnx --calib-path test/images/ --yaml hailo_model_zoo/hailo_model_zoo/cfg/networks/yolov8s.yaml --classes 2 --hw-arch hailo8l

Note: Using the following yolov8s.alls works fine but when i increase the layers it gives that error.

normalization1 = normalization([0.0, 0.0, 0.0], [255.0, 255.0, 255.0])
change_output_activation(conv42, sigmoid)
change_output_activation(conv53, sigmoid)
change_output_activation(conv63, sigmoid)
quantization_param(conv35, precision_mode=a16_w16)
quantization_param(conv46, precision_mode=a16_w16)
quantization_param(conv57, precision_mode=a16_w16)
nms_postprocess("../../postprocess_config/yolov8s_nms_config.json", meta_arch=yolov8, engine=cpu)

I have verified from model-report these 3 layers get 16-bit weights. BUT on increasing layers it starts getting error.