Can DFC oompile ONNX model trained with QAT from pytorch

General
1

Now I succeed to compile our own yolov5m model trained in pytorch to hef, but the detection accuracy is lower than the tensorRT engine. To improve the accuracy, can we compile the model with onnx model trained with QAT from pyTorch? Is there any instruction and sample?

Thanks a lot.

3

Hi @dliu,

The QAT would need to be performed as part of Hailo’s conversion process. You can read more about Hailo’s QAT here.

It is important to note that this is a fairly preview feature and is still limited in usage.

4

Thank you Marinavb, I read the document, and found one line “Currently, Hailo QAT only supports Keras”. So, I cannot use QAT with pyTorch.

Do you know any plan for Hailo to support QAT with pyTorch?

Thanks,
Dan

5

That will only have a negative impact if your model has operations unsupported on the device that are hard to implement in Keras. Is that the case?

Yes, but it won’t be added in the near releases.

6

I’m sorry that I don’t have any experience on TensorFlow and Keras. I started to study on it. Maybe I can try to train our yolov5m model with keras and use hailo QAT.

7

If the model is yolov5m then indeed it is problematic to do it with Keras.

Are you facing accuracy degradation after quantization? Have you tried some other post quantization algorithms that may help?

8

Sorry for my late response. Yes, we get accuracy degradation after quantization. We have succeed to compile our yolov5m model ( we modified with some new categories) from onnx to hef. But when we compare the accuracy from tensorRT, we found it drops obviously. I’m trying to find a way to improve the accuracy at least to tensorRT (without QAT) level. So, if you think it is problematic to do even with Keras for yolov5m, do you have other advice I can try?

9

A few things that could help:

  1. Use the same model script used in the model zoo
  2. Ensure the calibset is a good representation of the inference dataset
  3. Go through the accuracy debugging flow in our guide.