Hello,
Which layers it would be recommend to set 16 bit for yolov8m model for obj detection to increase prediction accuracy? I have tried: [“conv58”, “conv71”, “conv83”], but got an error:
[default] [info] Value 86,440 is not possible to represent in 2s complement using 13 bits. Please check quantization values for node: conv57.
thank you!
Welcome to the Hailo Community!
You should not need to use 16-bit layers for Yolov8m. Did you compare the Yolov8m model in the Model Zoo with yours? What accuracy for full precision and quantized do you get for your model?
Hi Klausk, I have pretty decent accuracy already with my custom model, but purely from experiment perspective would like to see the delta to make a judgement call regarding trades offs. I used these layers([“conv57”, “conv58”, “conv70”, “conv71”, “conv82”, “conv83”]) to make 16-bits and model optimization and compilation went “fine” meaning without errors, but model failed to predict anything as a result…