taOS Your AI, your hardware, your data. Self-hosted AI where your agents' memory, conversations, and files live on machines you own, fully offline when you want, cloud only when you choose.

user421 · July 11, 2026, 1:59am

Bonjour je viens de découvrir un excellent projet

Nous avons besoin de votre aide pour le développement sur raspberry pi 5 et Hat+2 40tops hailo 10H

github.com/jaylfc/taOS

Pi 5 + Hailo-10H (AI HAT+2, 40 TOPS): wire the hailo-genai LLM backend (community help wanted)

opened 03:42PM - 10 Jul 26 UTC

jaylfc

help wanted feature future models backend-driven catalog kilo-triaged kilo-auto-fix

Tracking issue for accelerated LLM inference on the Raspberry Pi 5 with the Hail…o-10H (AI HAT+2, 40 TOPS). Community help wanted from anyone who has this exact hardware. ## Primary goal: zero-touch, auto-detected + auto-installed (parity with the RK3588 NPU) The bar is the same as the Rockchip NPU experience: the user should NOT have to hand-configure anything. On install, taOS should detect the Hailo accelerator and set up its driver + runtime + backend automatically, exactly like it does for RK3588 today: - **Detection at install time.** `install.sh` already gates the Rockchip path on the device node (`if [ -e "/dev/rknpu" ]`) and runs the RK NPU setup. Mirror that: detect the Hailo device (e.g. `/dev/hailo0` / the PCIe device) and, if present, run a Hailo setup path. - **Auto-install the stack.** RK3588 installs its runtime via `scripts/install-rknpu.sh` (+ `install-ezrknpu.sh`). Add the equivalent `scripts/install-hailo.sh`: ensure HailoRT + `hailo-genai` are present (install if missing), matching the "install just works" principle. - **Runtime detection.** `tinyagentos/hardware.py` and the worker`s `detect_backends()` (`tinyagentos/worker/agent.py`) detect the RK3588 NPU and register it as an accelerator. Add Hailo there so a Hailo host auto-registers with an LLM-capable backend instead of falling back to CPU. - **Managed backend service + catalog**, same one-tap pattern the other accelerators use. Net: plug in the AI HAT+2, run the taOS installer, and Hailo-accelerated LLMs just work, no manual steps. ## Where things stand today - The verified, daily-driven NPU backend in taOS is Rockchip RK3588 (Orange Pi), via rkllama/rknn. That path is solid and is the reference implementation to mirror. - The Pi 5 + Hailo-10H is documented as a target profile (`arm-hailo10h-8gb` / `arm-hailo10h-16gb` in `os-build/boards/rpi5.md`) and the Hailo-10H is genuinely LLM/VLM-capable (unlike the vision-only Hailo-8L), but the board status is still "Planned" (`os-build/README.md`). There is no taOS Hailo LLM backend wired yet. - Until then, taOS runs on the Pi 5 fine in CPU mode via llama.cpp (e.g. Qwen3-4B Q4_K_M at ~4-6 tok/s on 8GB). The Hailo excels at vision right now. ## Work breakdown 1. Detection: recognise the Hailo-10H (device node + HailoRT) at install time and at worker startup, so it auto-registers as an accelerator (parity with `/dev/rknpu`). 2. Auto-install: `scripts/install-hailo.sh` to ensure HailoRT + `hailo-genai` (mirror `install-rknpu.sh`). 3. Runtime: drive LLM inference through `hailo-genai` (Hailo`s GenAI runtime for LLMs/VLMs on the -10H). 4. Model format: the HEF (Hailo Executable Format) conversion path for a couple of small chat models + catalog manifests for the Hailo variants (mirroring the RK3588 RKLLM variants). 5. Managed service + one-tap install, matching the backend-service pattern. ## How to help (you have the hardware) If you have a Pi 5 + Hailo-10H with the Hailo stack installed (HailoRT + hailo-genai), you are perfectly placed to help: - Comment here with your setup: Pi OS vs Armbian, RAM, HailoRT + hailo-genai versions, and how you installed them (that directly informs the auto-install script). - Share what LLM inference you can get running directly with hailo-genai (which models, tok/s), independent of taOS. That establishes the ground truth we build the backend against. - Test taOS builds/branches as the backend takes shape and report back. - Contributions welcome via PR if you want to dig into the detection + backend adapter. Reference: HailoRT + hailo-genai docs, and the existing RK3588 backend + `install.sh`/`install-rknpu.sh`/`hardware.py`/`worker/agent.py` detection in taOS as the exact shape to mirror. Thanks in advance to anyone who jumps in.

Soyez nombreux ne soyez pas timide même le fait de tester et de reporter les buts aides la communauté…

Merci de vos retours