Qwen compiled LoRA hef successful but not possible to create LLM object via hailo_platform.genai

General Discussion
1

I compiled Qwen2-1.5B LoRA with DFC 5.2.0 using the official v5.2 HAR, optimization ALLS and compilation ALLS. Compilation succeeds and hailortcli parse-hef succeeds. The HEF contains router__prefill/router__tbt, engineer__prefill/engineer__tbt, responder__prefill/responder__tbt. However hailo_platform.genai.LLM(vdevice, hef, "router") fails with HAILO_HEF_FILE_CORRUPTED(91), while the official non-LoRA HEF loads successfully. Does LLMWrapper support custom multi-LoRA HEFs, or does the HEF require additional GenAI metadata / a different packaging step?

1 Like
3

Alternatively, have you considered checking if the low-level HailoRT API (hailo_platform outside the GenAI wrapper) allows you to directly control the activation of the named network group?

Since hailortcli parse-hef validates the file, the HEF itself is structurally sound. The HAILO_HEF_FILE_CORRUPTED(91) error is likely a false positive triggered by the high-level GenAI wrapper parsing logic, which might expect a rigid, single-LoRA naming convention (e.g., hardcoded lookups for prefill and tbt without custom prefixes).

Bypassing the genai abstraction layer and dropping down to the core hailo_platform API allows you to orchestrate the multi-LoRA switching manually by activating the specific network groups directly.

Here is a conceptual implementation of how you can handle the custom LoRA multiplexing in low-level Python:


from hailo_platform import VDevice, Hef, ConfigureParameters, HailoStreamInterface

# 1. Raw HEF loading (should succeed since the file structure is valid)

hef = Hef("qwen2_multilora.hef")

vdevice = VDevice()

# 2. Configure network groups

configure_params = ConfigureParameters.get_default_configure_parameters_by_name(hef, interface=HailoStreamInterface.PCIe)

network_groups = vdevice.configure(hef, configure_params)

# 3. Explicitly target and activate the "router" LoRA for the Prefill phase

router_prefill_group = next(ng for ng in network_groups if ng.name == "router__prefill")

with router_prefill_group.activate():

    # Send prompt tokens to the device bindings

    # ... read/write stream operations ...

    print("Router Prefill network group activated and executed.")

# 4. Switch context to the "router" Token-by-Token (TBT) generation phase

router_tbt_group = next(ng for ng in network_groups if ng.name == "router__tbt")

with router_tbt_group.activate():

    # Execute the autoregressive token generation loop

    # ... read/write stream operations ...

    print("Router TBT network group activated for token generation.")

If you manage to interact with the network groups using this low-level workflow, it confirms that the issue lies entirely within the LLMWrapper metadata assumptions.