TF Resnet50v1 accuracy degradation

General
,
1

I fine-tune a ResNet50v1 model, but removing last layer and adding
Dense(128, ‘relu’ ) and output layer((number_of_classes), softmax) in TF2.15.1, for target hailo8l.

i had about 87% accuracy.
By converting to TFLITE and quantising model with opt=3, compression=0, without any added pre-processing, i got about 25-30dB in SNR.

The emulator running SDK_FP_OPTIMIZED, i get expected accuracy around 88-89%. But switching to SDK_QUANTIZED, i get accuracy drop down to less than 40%.

i noticed hailo_model_zoo/hailo_model_zoo/cfg/alls/generic/resnet_v1_50.alls at master · hailo-ai/hailo_model_zoo · GitHub, and tried this, but didnt help. and then found hailo_model_zoo/hailo_model_zoo/cfg/alls/hailo8/base/resnet_v1_50.alls at master · hailo-ai/hailo_model_zoo · GitHub which has a lot more parameters and not sure why.

not sure how to debug and resolve this to fix the accuracy . my model does not differ from the one from hailo model zoo.

3

Hey @jka21,

Can you run the following:

hailo analyze-noise model.har --data-path path_to_data
hailo parse-hef model.hef

Also some things to make sure you did:

Ensure Correct Calibration Dataset

  • Use 1024+ representative images.
  • Use the Hailo Model Zoo tools or SDK API to create a TFRecord for calibration.

Also can you provide the .alls file you trained your model with?

4

hello,

issue was on my side on the pre-processing in the end.
in the end all models including resnet showed small accuracy loss, even with compression=2. but had to use optimization=3 in all cases.
only in cases where there were a lot of layers (e.g. densenet201) i saw big loss up to 8% which i guess i can use better opt modes to resolve.