Windows 11 vs Hailo-8 accelerator

General
1

Ok, Hailo is exactly like Linux a trouble shooting based clean piece of hardware. I love it.

Hailortcli driver works like a charm in Windows.

Used separately = the docker via Docker.Desktop.

The only thing what it does is put .onnx → .has → .hef via the docker and setting up some kind of host…

Then the troubleshooting begins letting it run trough windows.

My goal is pure inference GPU/CPU…HAILO…CPU.

And a software would be hot since you guys creators are in the development.

Help would be hot this piece of equipment is a love hate relationship.

2

Hey @kevin_drouen,

To better assist you with your project, it would be really helpful if you could provide some additional details about the type of model you’re working with and the specific application you’re developing.

Also, if you could clarify your desired pipeline and how you envision the different components (GPU, ONNX Runtime, Hailo) fitting together, that would give me a clearer picture of what you’re aiming to achieve.

If you’re interested in using a GPU or ONNX Runtime in combination with Hailo, we actually have a great example that demonstrates how to set that up. You can find it here:

Feel free to take a look and see if it aligns with what you have in mind.

I’m excited to help you bring your vision to life, but to do that effectively, I could use a bit more context about your specific goals and requirements.

Looking forward to learning more about your project so we can dive in and find the best solution together!

Best regards,
Omria

3

Hi, and thnx for your response.

Ok watch:

For Windows:

Hailortcli (HailoRT)

Windows Ubuntu WSL:

Hailo compiler

Hailo model zoo

Driving on:

I5 13e gen

7900xtx with the ROCm install.

Hailo-8 (wich goes on a raspberry pi is now in my PCIe board with a NVMe port m.2).

It installs and shows up in my pc not as a stranger but as the the driver says.
I made the scanners for data on Windows, wich does this:

Data samples for 20 minutes, and that goes for

  • Network
  • GPU
  • CPU

It spits out in data model. After that i train it as a Network.h5, GPU.h5, CPU.h5.

After that i made from those 3 a combined model to h5.

Then i realized onnx and h5 could be a thingy so i made the h5 a .Keras.

Then i made the keras .onnx.

So what i dit is i putted that in the shared Ubuntu folder because Ubuntu and windows with the HAILO applications dont talk well bad pipelines.

Once done i tryed to make a has and from has to hef so that that i can script that the HAILO can interfere with my gpu and cpu like a captain on a ship telling them whats best for them to get their full potential.

I am stuck of making that kinda hef, i have made it once but that was on a hosted program from hailo in the suite but the suite is a very big bottleneck an needs his own OS that would be beautiful to throw that on WSL.

But for starters how the hell can i make a hef file with the compiler or model zoo i tryed endless options but none of them work very very annoying. I will show you one:

This is in Ubuntu WSL (II):

hailo optimize
–ckpt ~/hailo_env/combined_model.onnx
–model-script ~/hailo_env/model_script.alls
–output-har-path ~/hailo_env/optimized_model.har
–calib-set-path ~/hailo_env/calibration_data
–hw-arch hailo8l
–optimization-level 2
–compression-level 1

And this is a clean one:

hailo parser onnx /home/kevin/hailo_env/combined_model.onnx --output-har /home/kevin/hailo_env/combined_model.har --hw-arch hailo8l

One of the problems is that WSL cannot onder whatever conditions communicate, Windows with Ubuntu WSL. Why is that a problem? The compiler and model zoo cannot see the HAILO m.2 because it is in Windows…

4

Ok after tweaking and troubleshooting i have it done. It doenst take IP’s to tweak to make the keras go for the preparation to onnx.

For you challengers out there here you go before making an onnx in full then go to: hailo dfc-studio and make your has then optimize it to for hef then make the hef. But first the prep to onnx:

import onnx
import onnxruntime as ort
import numpy as np
import os

Stap 1: Modelpad instellen

MODEL_PATH = “/path/to/your/model.onnx” # Vervang dit met jouw ONNX-bestand
INPUT_SHAPE = (1, 3, 224, 224) # Pas aan voor jouw model
DEVICE = “CPUExecutionProvider” # Kies ‘CUDAExecutionProvider’ voor GPU

Stap 2: Controleren van het ONNX-model

def verify_onnx_model(model_path):
print(“Verifieer het ONNX-model…”)
try:
model = onnx.load(model_path)
onnx.checker.check_model(model)
print(“Model is geldig en compatibel!”)
except Exception as e:
print(f"Fout in het model: {e}")
return False
return True

Stap 3: Inferentie testen met ONNX Runtime

def test_onnx_runtime(model_path, device, input_shape):
print(f"Test inferentie met ONNX Runtime op {device}…")
session = ort.InferenceSession(model_path, providers=[device])
input_name = session.get_inputs()[0].name

# Random input genereren (dummy data)
input_data = np.random.rand(*input_shape).astype(np.float32)
result = session.run(None, {input_name: input_data})

print(f"Inferentie geslaagd! Outputshape: {result[0].shape}")

Stap 4: Hailo voorbereiding (optioneel)

def prepare_for_hailo(model_path):
hailo_model_path = os.path.splitext(model_path)[0] + “_hailo.hef”
print(f"Converteer {model_path} naar Hailo-formaat ({hailo_model_path})…“)
os.system(f"hailo_model_compiler --input_model {model_path} --hef {hailo_model_path}”)
print(“Conversie voltooid!”)
return hailo_model_path

Uitvoering

if name == “main”:
if verify_onnx_model(MODEL_PATH):
test_onnx_runtime(MODEL_PATH, DEVICE, INPUT_SHAPE)
print(“Start Hailo-voorbereiding…”)
hailo_hef = prepare_for_hailo(MODEL_PATH)
print(f"Hailo-HEF-bestand opgeslagen op: {hailo_hef}")

After this go to hailo dfc-studio the make the has file and follow the instructions.

Then optimize the has

Then compile the has to hef