InferModel set_buffer always sends 0 bytes on Raspberry Pi 5 aarch64 - HailoRT 4.23.0

General Discussion
1

Hi,

I’m running SuperPoint model inference on Raspberry Pi 5 with Hailo-8L using Python API, but set_buffer() always results in “Input buffer size 0” error regardless of approach tried.

Setup:

  • Raspberry Pi 5, aarch64, Raspberry Pi OS Bookworm
  • HailoRT 4.23.0 (hailort_4.23.0_arm64.deb)
  • Python binding: hailort-4.23.0-cp311-cp311-linux_aarch64.whl
  • Python 3.11.2
  • Hailo-8L M.2 module
  • HEF compiled with DFC 3.33.0 / HailoRT 4.23.0

Error:
[HailoRT] [error] CHECK failed - Input buffer size 0 is different than expected 307200

Input data:

  • numpy array shape (480, 640, 1), dtype uint8
  • C_CONTIGUOUS: True, nbytes: 307200

What I tried:

  1. InferVStreams with send() - same error
  2. InferModel + create_bindings + set_buffer + run_async - same error
  3. InferModel + create_bindings + set_buffer + C++ run() directly - same error

Note: CLI “hailo run model.hef --input-files input.bin” works at 129 FPS.
The issue is only in Python API on aarch64 RPi5.

Is there a known workaround for set_buffer on aarch64/RPi5?
Any working Python inference example for custom HEF on RPi5 would be appreciated.

Thank you

2

Update - Additional Debug Info:

Tried the minimal code pattern from community suggestions:

with VDevice() as vdevice:
infer_model = vdevice.create_infer_model(hef)
infer_model.set_batch_size(1)
with infer_model.configure() as configured:
bindings = configured.create_bindings()
input_data = np.zeros(infer_model.input().shape, dtype=np.uint8)
bindings.input().set_buffer(input_data)
configured.run([bindings], 10000)

Debug output before run():
INPUT SHAPE : [480, 640, 1]
INPUT FORMAT : FormatType.UINT8
INPUT NBYTES : 307200 ← correct, 480x640x1

Error still occurs:
CHECK failed - Input buffer size 0 is different than expected 307200

Confirmed:

  • set_batch_size(1) called before configure()
  • input().shape returns [480, 640, 1] — no batch dim
  • input_data.nbytes == 307200 — correct size
  • C_CONTIGUOUS == True
  • Tested with HailoRT 4.20.0 and 4.23.0 — same error both versions
  • CLI “hailo run model.hef --input-files input.bin” works at 129 FPS

This confirms the issue is in the Python binding set_buffer()
not transferring the numpy buffer pointer to C++ on aarch64.

3

Hi @eka_wijaya,

This is a numpy 2.x ABI incompatibility with the HailoRT aarch64 wheel. The C++ binding silently gets a null pointer when extracting the buffer.

Try to fix with:

pip install numpy==1.26.4

The HailoRT 4.23.0 wheel was built against numpy 1.x. Numpy 2.0 changed the C array interface, breaking set_buffer() on aarch64.

Please see also: