A question about quantized

General
1

What is the meaning of the quanzied number, most of model is positive:

image
image1120Ă—708 132 KB

But there is negative number:
image
So, what is the meaning of it?

2

Hey @jiahao.li

In this context, the “Quantized” number represents the degradation in model performance (often measured in terms of accuracy or mAP) due to quantization. Quantization is a technique where model parameters are reduced in precision (e.g., from 32-bit floating-point to 8-bit integers), typically to make the model run faster and use less memory. However, this process can cause a slight reduction in accuracy.

Here’s how to interpret the values:

  • Positive Values: These indicate the degradation percentage due to quantization. For example, a value of 1.41 means the model’s accuracy decreased by 1.41% after quantization. This is generally an acceptable trade-off for the efficiency gained.

  • Negative Values: A negative quantization degradation number, such as -2.54, implies that the model actually improved in accuracy after quantization. While rare, this can happen due to reduced overfitting or other quantization-related effects that, in some cases, can enhance model performance slightly.

In summary:

  • Positive quantized numbers = accuracy loss due to quantization.
  • Negative quantized numbers = accuracy improvement post-quantization.
1 Like
3

Thank you very much for your explanation.