ConvNext on Hailo8 Non-Deterministic HEF Results

[lawrence]

I don’t think its specific to LayerNorm, I’ve experienced this with fast-plate-ocr and also yolov8 (w/ yolov8 the differences aren’t enough to affect the end results but they are not small fp differences like stated in the documentation).

Trying to Add Hailo Support to fast-plate-ocr - Inference Output Looks Wrong

@Andy_Andrw @omria I did some personal digging as well. I compiled the hef and also ran emulation for the native model, the optimized floating point model, and the quantized model. The native model and the optimized floating point model gives the same probits and postprocessing them gets the same answer as an onnx inference. The quantized emulation does not give the same probits but post processing gets the same (correct) answer as an onnx inference. However, inference using the hef does not …

You can see the difference here with fast-plate-ocr. My comment also links to two different topics that also experience the same issue (yolov5 and some unknown network).