Qwen3:1.7b repeats instead of copying a short label (Hailo-10H, HailoRT 5.4)

General Discussion
, ,
1

Hi everyone,

I’m using qwen3:1.7b on a Raspberry Pi 5 with Hailo-10H and getting repeated words when I ask it to copy a short label. I’d appreciate help checking whether this is expected with greedy decoding or whether something in my setup is wrong.

My setup is Debian 13/Trixie, with HailoRT, PCIe driver, firmware and GenAI Model Zoo all at 5.4.0. Package integrity checks passed.

Here is the request to Hailo-Ollama’s /api/chat, reconstructed from my client’s recorded parameters and adapter code:

{
  "model": "qwen3:1.7b",
  "stream": false,
  "messages": [
    {"role": "system", "content": "Copy the requested label exactly, preserving every word and its order. Output only the label, without quotation marks or explanation."},
    {"role": "user", "content": "Label: red red red blue red red red blue."}
  ],
  "options": {
    "temperature": 0.0,
    "num_predict": 128,
    "frequency_penalty": 1.0,
    "top_p": 0.8,
    "top_k": 20,
    "seed": 42
  }
}

Expected:

red red red blue red red red blue.

The actual response starts with:

red red red red red red red blue red red ...

and keeps repeating until the length limit. The fourth word is already red instead of blue.

Changing only frequency_penalty to 1.05 returns the eight correct words and stops, but drops the final period. In separate Python SDK tests with context cleared, sampling enabled, T 0.7 or 1.0, p 0.8, k 20, seed 42 and penalty 1.0, both runs copied the label including the period. I haven’t yet run this exact greedy case directly through the SDK, so I haven’t ruled out the HTTP path.

There is also a model-name detail I don’t understand. The file is Qwen3-1.7B-Instruct.hef (2,880,748,478 bytes), but its embedded configuration and network names say Qwen3-VL-2B-Instruct. parse-hef reports compiler 5.3.0. Its SHA-256 matches the official checksum I checked:

cc9b9d1c92e35249b5a9b7bc31fbd652f03bba1232e99b9a8271845ad6f17821

Could someone from Hailo help with these two points?

  1. Can you reproduce this copy failure, and which decoding settings are recommended for this HEF? Is there a known issue with copying or repetition?
  2. Is the qwen3:1.7b tag intentionally the text decoder of Qwen3-VL-2B-Instruct? Which source checkpoint/revision does this file use, and is it supported on the 5.4 stack?

I have the complete outputs, parameter records and HEF quantization metadata available if useful. Thank you.