Lightweight HailoRT bindings in Nim – simple, fast inference without Python or CMake

General Discussion
1

Hi,

I’ve been working with HailoRT and felt that:

  • Python bindings are convenient but sometimes too heavy
  • C++ works but build setup (CMake, etc.) can get painful
  • Simple inference requires quite a bit of boilerplate

So I built a minimal Nim binding + high-level Detector API.

Example:

import hailort_nim

let det = Detector.open(“model.hef”).get
let res = det.detectNmsByClass(input).get

for d in res:
  echo d.classId, " ", d.score

It uses Futhark to generate bindings from the C API, so no manual wrapping.
Tested with YOLOv11s HEF models, including NMS parsing.

Repo: GitHub - centurysys/hailort_nim: Minimal and fast Nim bindings for HailoRT with a simple Detector API · GitHub

Performance

Tested on AM67A (Cortex-A53 1.4GHz), single inference:

Python:

real    1.592s
user    1.389s
sys     0.361s

Nim:

real    0.465s
user    0.075s
sys     0.308s

In this setup, Python spends most time in user-space processing,
while Nim is mostly bound by device I/O.

(~3x faster in this simple test)
Would love feedback, especially from people who tried Python/C++ approaches.

2 Likes
2

Thanks @Takeyoshi_Kikuchi for sharing this useful project - We appreciate it!

3

Hi,

I’ve been working on improving my hailort_nim bindings and wanted to share an update.

Recent changes:

  • Multi-HEF support using openPrepared / activate / deactivate
    → allows multiple models to stay loaded and switch with low overhead (~2–3 ms activate)
  • Device status APIs (temperature, throttling)
  • Power measurement is now handled as optional since it’s not supported on all boards
  • Fixed a critical ownership issue where physical devices from vdevice were incorrectly released,
    which could lead to segmentation faults

This makes the library much more stable for long-running applications.

Repo:

Feedback is welcome!

1 Like
4

Thanks @Takeyoshi_Kikuchi !