The Whisper-Small-genai.hef from the GenAI Model Zoo v5.2.0 ( hailo_model_zoo_genai/docs/MODELS.rst at v5.2.0 · hailo-ai/hailo_model_zoo_genai · GitHub ) does not reliably respect the language parameter in the Speech2Text API. Even with language=“de” and task=Speech2TextTask.TRANSCRIBE, the model frequently outputs English instead of German. The Whisper-Base-genai.hef does NOT have this issue.
Environment
- Hardware: Raspberry Pi 5, 8GB + AI HAT+ 2 (Hailo-10H)
- HailoRT: 5.2.0
- Python: 3.13
- HEF files: Downloaded from hailo.ai/developer-zone (GenAI Model Zoo 5.2.0)
- Whisper-Base-genai.hef (131 MB) — works correctly
- Whisper-Small-genai.hef (388 MB) — language bug
- OS: Debian 13 (Trixie), aarch64
Reproduction
import numpy as np
from hailo_platform import VDevice
from hailo_platform.genai import Speech2Text, Speech2TextTask
vdevice = VDevice()
s2t = Speech2Text(vdevice, “/path/to/Whisper-Small-genai.hef”)
# audio_16k: German speech as float32, 16kHz, mono
text = s2t.generate_all_text(
audio_data=audio_16k,
task=Speech2TextTask.TRANSCRIBE,
language=“de”,
timeout_ms=30000,
)
Often outputs English instead of German!
Test Results
Test audio: German TTS (Piper de_DE-thorsten-medium), resampled to 16kHz float32 mono.
Whisper-Small GenAI (language=“de”, task=TRANSCRIBE):
Input: “Ich moechte gerne einen Termin vereinbaren.”
Output: “I would like to have a Termin Vereinbarer.” — English/mixed, WRONG
Input: “Wann haben Sie geoeffnet?”
Output: “When did they open?” — English, WRONG
Input: “Koennen Sie mich bitte zurueckrufen?”
Output: “koenne Sie mich bitte zurueckruven?” — German but with errors, PARTIALLY OK
With task=TRANSLATE all outputs are English (expected).
Whisper-Base GenAI (same API, same parameters) — works correctly:
Input: “Ich moechte gerne einen Termin vereinbaren.”
Output: “Ich moechte gerne einen Termin vereinbaren.” — CORRECT
Input: “Wann haben Sie geoeffnet?”
Output: “Wann haben Sie geoeffnet?” — CORRECT
Whisper-Base correctly respects language=“de” every time.
Analysis
According to MODELS.rst ( hailo_model_zoo_genai/docs/MODELS.rst at v5.2.0 · hailo-ai/hailo_model_zoo_genai · GitHub ), both HEFs are compiled from the multilingual HuggingFace models (openai/whisper-base and openai/whisper-small), not the .en variants.
Since the Speech2Text API offloads “tokenization and embeddings to the Hailo-10H completely” (as noted in the forum), the language token is processed on the NPU. The Whisper-Small quantization/compilation appears to have broken the language token handling, causing the model to default to English regardless of the language parameter.
Expected Behavior
language=“de” with task=Speech2TextTask.TRANSCRIBE should consistently produce German transcription, as it does with Whisper-Base GenAI.
Current Imperfect Workaround 
Using Whisper-Base-genai.hef instead. On 8kHz telephone audio we measure:
- WER: 15.7% (German)
- Latency: 424ms average
- language=“de” works reliably