Hi,
I compiled 2 different models of Hailo on 2 different datasets. One dataset has very low lightning conditions, other dataset has good lightning condition.
I had to use optimization level 4, with 1024 images to get the best performance. However, still I get some performance drop. I made sure that I cover all types of defects to create 1024 calibration dataset.
1 model gives me 1.5% drop compared to the original pytorch model accuracy, and the other model with low lightning condition gives me 8% drop on Hailo.
Why am I getting results like this? With same optimization setup, ideally both the models should give similar performance. Adaround optimization with greater than 1024 images, takes huge amount of time, and is ideally not feasible.
Both the models are exactly same custom models. Also, the image resolution of both the dataset is same.