FLOAT32 Custom HEF model problem

General
,
1

Hi everybody,
I am converting my FLOAT32 input and output onnx model to a HEF format model. (With CLI or Python SDK)

The model I converted automatically converts to UINT8 input/output.
My goal is to give FLOAT32 input and get FLOAT32 output.
With nms_postprocess I think it is possible to get the output as FLOAT32 in certain models.
However, all the examples I looked at take yaml/json as a parameter and I could not understand what this file is, or rather how will I prepare it for my own model.

Question: Can I give FP32 input/output to a model converted from FP32 ONNX to HEF?

2

The Hailo-8 processes values in integer formats (4-bit, 8-bit [default], or 16-bit). HailoRT can quantize and dequantize both input and output values. For more information, refer to the inference tutorials available in the Hailo AI Software Suite Docker or the HailoRT User Guide.

NMS (Non-Maximum Suppression) post-processing has been added as an option for certain popular models, such as those in the YOLO family. This step runs on the host as part of HailoRT, includes dequantization, and therefore outputs data in FLOAT32. The corresponding JSON configuration files are tailored specifically to these models and do not represent a general solution for all model conversions.

Post-processing code is available for the models included in the Model Zoo, as well as in the Tappas examples. This code is designed to run as part of your application:

GitHub - Hailo Model Zoo - Postprocessing

GitHub - Tappas - Postprocesses

If you are using a custom model, you will need to implement the pre-processing and post-processing logic yourself and ensure it runs within your application.

3

Thanks for your answer.

So can I do nms post process on my own custom model,
same way can I do preprocess and give float32 input directly to the model?

4

If your model is a custom YOLO model that is compatible with our post-processing you can add NMS using the nms_postprocess model modification command. See Hailo Dataflow Compiler User Guide.

Yes, HailoRT can do the quantization for you. The model (HEF) on the Hailo device will receive quantized data.