Can't Optimize or Compile YoloV8 with trained on Custom Dataset

[Nadav]

Check this thread:

Problem With Model Optimization

I might have a solution, but I’m not sure if it’s general, but I think that it worth checking. It seems that on datasets that have low number of classes and/or low number of pictures, some nodes on the YOLOv8 are getting almost nullified. I’ve used this command to force a wider dynamic range on the outputs: quantization_param([conv42, conv53, conv63], force_range_out=[0.0, 1.0]) This is relevant for YOLOV8s trained from our retraining dockers. These are the end nodes before the NMS layer.