Options to convert many models

General
1

Hello team,

After little success with using the SDK on an large GPU instance (Ubuntu EC2 instance) to convert a test model (yolo8n small)), I wanted to understand if anyone has actually created a pipeline for such.

On the ec2 instance, we convert from .pt to onnx just fine, but sadly the 2nd .hef conversion/optimisation hangs and fails (usually noticing gpu not found errors). On checking with the docker host and image, we can see the GUP librarys are available.

Ideally, for us to consider H8’s we will need such a pipeline thats easy to use and easy to deploy, I can’t say I have not been going round in circles for the last 24 hours trying to just convert!

Frustrations aside, if someone would like to point me in the current right direction, I will happily continue to build such a pipeline.

2

Hey @james2 ,

Welcome to the Hailo Community!

ONNX to HEF Conversion Pipeline for Hailo

1. ONNX to HAR

Parse ONNX to Hailo Archive (HAR):

hailomz parse --ckpt model.onnx --hw-arch hailo8 --output model.har

2. Optimize HAR

Quantize and optimize HAR for Hailo hardware:

hailomz optimize --har model.har --calib-path /path/to/calibration/imgs/ \
  --output optimized_model.har

Optional flags:

  • --performance: Enable performance optimization
  • --classes 80: Adjust output classes
  • --resize 640 640: Auto-resize input

3. HAR to HEF

Compile optimized HAR to Hailo Executable Format (HEF):

hailomz compile --har optimized_model.har --hw-arch hailo8 --output model.hef

4. Profile Performance (optional)

Evaluate model performance pre-deployment:

hailomz parse model.onnx
hailo profiler model.har

5. Deploy on Hailo

Run HEF file on Hailo hardware:

hailortcli run model.hef

Summary Table of Key Steps

Step Command Documentation Reference
Convert PyTorch to ONNX torch.onnx.export(...) Model Zoo Guide, Page 17
Parse ONNX to HAR hailomz parse ... Model Zoo Guide, Page 17
Optimize HAR hailomz optimize ... Model Zoo Guide, Page 18
Compile HAR to HEF hailomz compile ... Model Zoo Guide, Page 19
Profile Model hailo profiler ... Model Zoo Guide, Page 18
Deploy on Hailo hailortcli run ... HailoRT Guide

Refer to Hailo’s Model Zoo User Guide and HailoRT User Guide for more details on each step. Let me know if you have any other questions!