Hi everyone,
I would like to deploy a quantization-aware-trained model on Hailo-8 using DFC 3.34.0. I’m trying to understand whether the quantization established during training can be preserved, and what deployment workflow Hailo recommends.
Training Setup
My TensorFlow/Keras 3 training setup supports QAT using TensorFlow’s fake-quantization operations:
-
Training maintains floating-point weights while simulating 8-bit quantization during the forward pass, with straight-through gradient estimation.
-
Convolution weights use symmetric quantization, per output channel for standard convolutions and per tensor for depthwise/transposed convolutions.
-
Activations use asymmetric, per-tensor quantization. Their ranges are tracked using exponential moving averages during training.
-
Activation range observers can be frozen towards the end of training and remain fixed during inference.
This is external QAT, not QAT performed through Hailo’s SDK.
Exported Model
The model is exported through tf2onnx. The resulting artifact is an ONNX Q/DQ model, representing quantization through QuantizeLinear and DequantizeLinear nodes with explicit scales and zero-points.
My initial assumption was that, because quantization has already been established during training, the deployment flow could be:
QAT-trained ONNX Q/DQ model → DFC parsing → compilation → HEF
Questions
- Can DFC import this external ONNX Q/DQ model and preserve its quantization parameters?
- If direct Q/DQ import is unsupported, what is the recommended conversion path? Would another format, such as TFLite, preserve the external QAT information, or does Hailo require a floating-point input model regardless?
Best regards,
Julius
