Hailo-10H working on kernel 6.18.34 (RPi 5) using PR #52 + HailoRT 5.3.0

Guides
1

Success on getting Hailo-10H working on Kernel 6.18.34 (Rpi5) (Trixie)

  1. It looks like the PR from here works. Grab it from Github
    1. Install build dependencies
      1. sudo apt update
      2. sudo apt install -y build-essential dkms git linux-headers-$(uname -r)
    2. Install the original 5.3 hailort driver + hailort usermode files from Hailo Developer zone. We’ll do the .deb files
      1. Download .deb files hailort_5.3.0_arm64.deb, hailort-pcie-driver_5.3.0_all.deb
      2. sudo dpkg -i hailort-pcie-driver_5.3.0_all.deb
      3. sudo dpkg -i hailort_5.3.0_arm64.deb
    3. Clone the hailort-driver repository and change to PR #52
      1. git clone GitHub - hailo-ai/hailort-drivers: The Hailo PCIe driver is required for interacting with a Hailo device over the PCIe interface · GitHub
      2. cd hailort-drivers/linux/pcie
      3. git fetch origin pull/52/head:pr-52
      4. git checkout pr-52
    4. Build the driver
      1. make clean
      2. make install_dkms
    5. Install the new driver
      1. sudo modprobe -v hailo1x_pci
    6. Verify that it looks good
      1. dmesg | grep -i hailo | tail -5
      2. sudo hailortcli scan
      3. sudo hailortcli fw-control identify
2 Likes
2

Hi user483,

I found your guide while troubleshooting the always failing install of /hailort-pcie-driver (5.3.0).

Checking the hailort-pcie-driver.deb.log showed the invalid function call del_timer_sync in file monitor.c.

My current fresh RPI install is Debian Trixie and kernel 6.18.34+rpt-rpi-2712

Reference: HailoRT Build Development Doc - warts and all

I looked for the file and edited all 3 instances. Then I extracted the postinst.sh from the deb-package, ran it and the install succeeded.

Once that was done I could follow your instructuions successfully.

Steffen

3

Great! The important thing is to get the changes from pull request 52.

4

Continuing my adventures with hailo-10H and raspberry pi, this is what I needed to do to get hailo-ollama working. One note: I’m not using the default port of 8000 as I’m using it for another service. So, you can probably skip those steps.

Setup:

  • HailoRT 5.3.0

  • Hailo-10H adapter

  • hailo-ollama running as a systemd service under a non-root user

  • Service configured with Environment=OLLAMA_HOST=0.0.0.0:11000

Symptoms:

  • The server started and responded to /api/version and /api/tags

  • /api/generate and /api/chat both failed with:

    • “LLM not loaded”

    • Failed to create VDevice

    • Failed to open device file /dev/h1x-0 with error 13 (Permission denied)

Root Cause:
The Hailo device node (/dev/h1x-0) was created with root:root permissions (mode 600). No hailo group existed, so the non-root user running hailo-ollama could not access the device.

Fix that worked:

  1. Create the group and add the user:

    Bash

    sudo groupadd hailo
sudo usermod -aG hailo loggedOnUserName

  2. Create the following udev rule:

    Bash

    sudo tee /etc/udev/rules.d/99-hailo.rules <<EOF
ACTION=="add", KERNEL=="h1x*", SUBSYSTEM=="char", GROUP="hailo", MODE="0660"
EOF

  3. Reload rules:

    Bash

    sudo udevadm control --reload-rules
sudo udevadm trigger --action=add

  4. (Temporary test) Fix permissions immediately:

    Bash

    sudo chown root:hailo /dev/h1x-0
sudo chmod 660 /dev/h1x-0

  5. Restart the service:

    Bash

    sudo systemctl restart hailo-ollama

After this, /api/generate and /api/chat started working normally.

Verification:

Bash

curl -s http://localhost:11000/api/generate \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen2:1.5b", "prompt": "Hello", "stream": false}' | jq

Note:
The udev rule above was the most reliable one after testing several variations. The key was including ACTION==“add”.

Hope this helps anyone else running into VDevice / permission errors with HailoRT 5.3.0 on Hailo-10H.

5

Continuing my adventures getting hailo-10 working with ollama, and running as the frontend for a hermes agent. Please note that the previously documented steps for 99-hailo.rules is incorrect. Below is better steps and configuration.

Troubleshooting hailo-ollama on Raspberry Pi 5 + Hailo-10H

Date: July 2026
Hardware: Raspberry Pi 5 (8GB) + Raspberry Pi AI HAT+ 2 (Hailo-10H)
Goal: Run a fast, always-on local LLM frontend using hailo-ollama with systemd

Introduction

The hailo-ollama project is a custom C++ port/fork of Ollama optimized for Hailo accelerators. While it provides excellent performance on the Hailo-10H, it has several important behavioral differences from the official Go-based Ollama. These differences can cause confusing failures when following standard Ollama documentation or assumptions.

This guide documents the issues encountered while setting up hailo-ollama as a systemd service with a dedicated model directory and persistent model loading (keep_alive).

During configuration, I moved hailo-ollama from running as the typical user, and moved it to a new hailo-ollama user. Most of these troubleshooting steps were failures with am llm trying to configure hailo-ollama correctly and required manual intervention.

The biggest problem of course is that LLMs will try to treat hailo-ollama as ollama.

Troubleshooting Ledger

Step Problem Root Cause Solution
1 Changes to environment variables (e.g. model path) did not take effect after systemctl daemon-reload daemon-reload only updates systemd’s configuration. It does not restart the running process or apply new environment variables to it. Run sudo systemctl restart hailo-ollama to fully stop and restart the service with the new environment.
2 systemd reported “Unknown key” for rate-limiting settings StartLimitIntervalSec and StartLimitBurst must be placed in the [Unit] section in modern versions of systemd. Moved both directives into the [Unit] section.
3 JSON parse error / HAILO_INTERNAL_FAILURE(8) after configuration changes The running service was still using old/broken model manifests and templates from the previous directory. Restarted the service to clear stale state, then performed a clean model pull.
4 HTTP 500 error when using curl -X POST … -d ‘{…}’ curl -d defaults to application/x-www-form-urlencoded. The oatpp web framework used by hailo-ollama expects raw JSON. Added the header: -H “Content-Type: application/json”
5 “Model not found” even though model files existed on disk Moving or copying model files manually broke the cryptographic manifests and symlinks that Ollama-style systems rely on. Triggered a fresh /api/pull so the daemon could rebuild proper metadata and symlinks.
6 hailo-ollama CLI crashed with port conflict when trying to pull a model Unlike upstream Ollama, the hailo-ollama CLI attempts to start its own web server during pull operations. It conflicts with an already-running daemon on the same port. Bypassed the CLI entirely and triggered model downloads by sending a JSON payload directly to the running daemon’s /api/pull endpoint via curl.

Key Lessons & Gotchas

  • Forks have different behavior: Do not assume hailo-ollama behaves exactly like official Ollama. The CLI in particular has different behavior around server startup and manifest handling.

  • systemd environment changes require a restart: systemctl daemon-reload is not enough when changing Environment= variables. You must restart the service.

  • Rate limiting directives belong in [Unit]: Modern systemd is strict about this. Placing them in [Service] causes them to be ignored.

  • Manifests and symlinks are fragile: Manually moving model directories often breaks internal linkages. Let the daemon rebuild them via /api/pull.

  • Use the API when the daemon is running: When hailo-ollama is already running as a service, prefer sending requests to /api/pull and /api/generate rather than using the CLI.

Recommended systemd Service

Here is the final working service configuration:

ini

[Unit]
Description=Hailo Ollama Server (Fast Frontend)
Documentation=https://hailo.ai/
After=network-online.target
Wants=network-online.target

StartLimitIntervalSec=60
StartLimitBurst=3

[Service]
Type=simple
User=hailo-ollama
Group=hailo-ollama

WorkingDirectory=/var/lib/hailo-ollama

Environment=OLLAMA_HOST=0.0.0.0:11000
Environment=OLLAMA_KEEP_ALIVE=-1
Environment=OLLAMA_MODELS=/var/lib/hailo-ollama/models

ExecStart=/usr/bin/hailo-ollama serve

Restart=on-failure
RestartSec=5

StandardOutput=journal
StandardError=journal
SyslogIdentifier=hailo-ollama

[Install]
WantedBy=multi-user.target

Key settings:

  • OLLAMA_KEEP_ALIVE=-1 → Model stays loaded for fast responses

  • Dedicated hailo-ollama system user + /var/lib/hailo-ollama directory

  • Rate limiting correctly placed in [Unit]

Additional Recommendations

  • Create a dedicated system user (hailo-ollama) rather than running as your personal user.

  • Use proper udev rules so the non-root user can access the Hailo device (/dev/h1x-*).

  • When the service is running, manage models via the HTTP API instead of the CLI to avoid port conflicts.

  • Monitor logs with: journalctl -u hailo-ollama -f

Conclusion

While hailo-ollama delivers strong performance on the Hailo-10H, it requires more careful systemd integration and a different operational approach than standard Ollama. The combination of proper service configuration, forced restarts after environment changes, and using the HTTP API for model management results in a stable, low-latency local inference setup suitable for always-on agentic frontends.

2 Likes
6

Since it wasn’t mentioned, the correct /etc/udev/rules.d/99-hailo.rules was

KERNEL==“h1x-[0-9]*”, GROUP=“hailo”, MODE=“0660”

The whole “ACTION==“add”” should not be used.

1 Like
7

Thanks for posting this. I did a few tiny tweaks to the git clone:

git clone http s://github.com/hailo-ai/hailort/ (stupid system here keeps expanding URLs)

and a sudo make install_dkms

But other than that up and running. Ollama is annoying, I’d really rather run a real version than Hailo’s with their dippy little models. The 10h should be able to run Gemma4 e4b or something at least with tools support.

Any thoughts on progress on that?

2 Likes
8

I’m glad I came across this post. Doing the exact project, wanted the hat to run Ollama with a Hermes agent frontend. I got 5.3.0 firmware going, but have a heck of a time setting up the service daemon and then Hermes.

I’ve started over a few times, still not done. Now 5.4.0 is out…

9

I found this for OpenClaw. Works with 5.3.0, just it does crash and you have to restart the proxy.