Accessing Quantization Offsets - Python SDK (4.19+)

General
1

Hi all,

I am using the Python Example for Object Detection and related code, and I was wondering how can I get the quantization offsets to preprocess the image? I see the InferModel class has a subclass with the quant_infos but when I inspect it, it only shows it is type property object and I have no idea how I can access the member.

The Python API documentation mentions it is a list, but I cannot iterate over it.

I already try to set the input type to FLOAT32, but I am wondering if it is doing input/output quantization as I cannot see the code (ig for obvious reasons).

Fyi: I am using a custom model that for business reasons, I cannot disclose. I can say it is a CNN.

Any help is welcome! Have a nice day!

2

Hi @Isaac_Nunez ,

Welcome to the Hailo Community!

The scale and zero point can be accessed in the following way:

  • Get the VStream info from the hef object:
    input_vstream_info = infer_model.hef.get_input_vstream_infos()
output_vstream_info = infer_model.hef.get_output_vstream_infos()
  • Access the scale and zero point values:
  for input_info in input_vstream_info:
      print(input_info)
      print("Scale: {}".format(input_info.quant_info.qp_scale))
      print("Zero point: {}'\n".format(input_info.quant_info.qp_zp))
  
  print("Outputs")
  for output_info in output_vstream_info:
      print(output_info)
      print("Scale: {}".format(output_info.quant_info.qp_scale))
      print("Zero point: {}\n".format(output_info.quant_info.qp_zp))

At runtime, you can set FLOAT32 format to the input/output streams if you want HailoRT to quantize/dequantize the data for you, or you can set integer format otherwise (and you may have to manually scale the data). For example, if you have a normalization layer at the input of the HEF, you typically set the input to UINT8.