Onnx -> hef conversion help

General
1

Subject: Assistance Required: ONNX to HEF Conversion Using HailoRT Source Installation

Hi everyone,

I am currently working on a project to convert an ONNX model to the HEF format for deployment on a Hailo-8 AI processor. I have made significant progress but am encountering some roadblocks and would greatly appreciate any guidance from the community.

Progress So Far

  1. System Setup:
  • Using a Raspberry Pi with Ubuntu and Hailo-8 PCIe card.
  • Installed HailoRT 4.19.0 from source.
  • Installed the PCIe driver (version 4.19.0) and confirmed successful firmware loading using dmesg.
  1. Tool Installation:
  • Successfully built the hailo-integration-tool from source.
  • Verified that the hailortcli tool is functional and lists the available commands (e.g., run, benchmark, etc.).
  1. Attempted Model Conversion:
  • Tried using the hailo-integration-tool to convert my ONNX model, but the tool seems focused on hardware tests and diagnostics rather than model compilation.
  • I was unable to locate a hailo_compiler tool in the HailoRT source package.
  1. Docker Installation Attempts:
  • Tried pulling the hailo/ai-software-suite:latest Docker image but received an “access denied” error.
  • Unable to locate or download the necessary .tar files for manual Docker installation.

Specific Goals

  • Convert my ONNX model (best.onnx) into the HEF format.
  • Run inference on the Hailo-8 device using the converted HEF file.

Issues Faced

  1. Missing Compiler Tool:
  • I cannot find the hailo_compiler tool (or equivalent Dataflow Compiler) in the HailoRT source installation.
  • It’s unclear if this tool is included in a separate package or Docker environment.
  1. Docker Access Challenges:
  • Unable to pull the hailo/ai-software-suite:latest Docker image due to repository access restrictions.
  • I lack access to download pre-built Docker .tar files for manual installation.
  1. Integration Tool Limitation:
  • The hailo-integration-tool appears to be designed for diagnostics rather than model compilation, and I couldn’t find any clear documentation to indicate otherwise.

Questions for the Community

  1. Where can I find the hailo_compiler (DFC tool) or an equivalent utility for ONNX-to-HEF conversion? Is it included in a separate package?
  2. Is there a way to convert ONNX models to HEF format using the HailoRT source installation, or is Docker a strict requirement?
  3. Can anyone clarify the purpose of the hailo-integration-tool for model compilation?
  4. How can I resolve Docker access issues to download the AI Software Suite or related resources?

I appreciate any advice or insights you can provide! Please let me know if more details are needed.

Thank you in advance for your help!

2

Please download the Hailo AI Software Suite Docker from our Developer Zone.
Just to make sure, the model conversion needs to be done on a high-end PC running Ubuntu and not on a small platform like the Raspberry Pi.

Converting a model to HEF is a process with multiple steps (parsing, optimizing and allocation/compilation) that require complex calculations and searches. It is done offline in the model build environment which can be different from the runtime.

The Hailo Integration tool is used to verify your hardware setup during design and production. There are different tests you can run:

  • confirmation of the number of PCIe lanes
  • measure PCIe bandwidth
  • test inference working
  • thermal tests that allow you to estimate the thermal conductivity of the heatsink and housing
  • stress test the system

You should have access to the Developer Zone. Log in, go to Developer Zone, Software Downloads, should be the first items on the page.