Qwen2-1.5B-Instruct-Function-Calling-v1 HEF fails with HAILO_TIMEOUT on HailoRT 5.2.0 — no v5.2.0 HEF available?

General Discussion
,
1

Hi,

I’m running a Raspberry Pi 5 + AI HAT+ 2 (Hailo-10H) with HailoRT 5.2.0 and trying to load the Qwen2-1.5B-Instruct-Function-Calling-v1 model.

The Problem

Loading the model via the Python GenAI API fails with HAILO_TIMEOUT(4):

[HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_TIMEOUT(4) - Failed to create LLM

[HailoRT] [error] Failed to create LLM with status HAILO_TIMEOUT(4). Check device logs for more details.

hailo_platform.pyhailort._pyhailort.HailoRTStatusException: 4

Code:

from hailo_platform.genai import VDevice, LLM

vd = VDevice()

llm = LLM(vd, “/opt/axon-agent/models/Qwen2-1.5B-Instruct-Function-Calling-v1.hef”)

What works

The base Qwen2-1.5B-Instruct.hef (downloaded from the v5.2.0 link) loads and generates perfectly on the same setup:


llm = LLM(vd, “/opt/axon-agent/models/Qwen2-1.5B-Instruct.hef”)

r = llm.generate_all([{“role”: “user”, “content”: “Hello”}], temperature=0.1, max_generated_tokens=10)

# Output: “Hello! How can I assist you today?”

Root cause (I think)

The [MODELS.rst]( hailo_model_zoo_genai/docs/MODELS.rst at main · hailo-ai/hailo_model_zoo_genai · GitHub ) lists Qwen2-1.5B-Instruct-Function-Calling-v1 as requiring HailoRT > 5.2.0, but the pre-compiled HEF is only available on the v5.3.0 link path:

So the HEF appears to be compiled with the v5.3.0 toolchain and isn’t backwards-compatible with the 5.2.0 runtime.

My environment

  • Hardware: Raspberry Pi 5 + AI HAT+ 2 (Hailo-10H)
  • OS: DietPi v10.2.3 (Debian Trixie, aarch64)
  • Kernel: 6.12.75+rpt-rpi-2712
  • HailoRT: 5.2.0
  • Driver: hailort-pcie-driver 5.2.0 (hailo1x_pci)
  • Firmware: 5.2.0
  • Python: 3.10.20 (pyenv) with hailort-5.2.0-cp310 wheel
  • Device confirms working: `hailortcli fw-control identify` reports Hailo-10H, firmware 5.2.0

Questions

  1. Is there a v5.2.0-compatible HEF for Qwen2-1.5B-Instruct-Function-Calling-v1?
  2. If not, when will HailoRT 5.3.0 packages be publicly available? (Currently only 5.2.0 is on the developer zone)
  3. As a workaround, is there a way to compile the function-calling model’s HEF using DFC 5.2.0 and the Qwen2-1.5B HAR?

Thanks for any help!

1 Like
2

Hi @user971 ,

In the upcoming days 5.3 will be available on our dev zone.

Thanks,

1 Like