HEF files for Whisper-base compatible with HailoRT 4.19.0 (Hailo-8, amd64)

General Discussion
1

Hello,

I am developing a ROS2-based voice assistant running on a Bosch Rexroth
ctrlX X7 industrial controller (amd64, Intel i7) with a Hailo-8 NPU.
The system runs ctrlX OS and is locked to HailoRT 4.19.0, which I cannot
update due to platform constraints.

I need HEF files for Whisper-base (encoder + decoder) compiled with
Dataflow Compiler 3.28.x, compatible with HailoRT 4.19.0.

I tried downloading pre-compiled HEFs from the Model Zoo S3 bucket
(v2.13.0) but the links return AccessDenied.

I also tried compiling the model myself using DFC 3.28.0, but I am
encountering parsing errors with the Transformer attention layers
(ew_mult expects 2 inputs but found 1).

Could you please:

  1. Provide pre-compiled HEF files for Whisper-base compatible with
    HailoRT 4.19.0 / DFC 3.28.x?
  2. Or point me to the correct S3 bucket path for Model Zoo v2.13.0?
  3. Or provide guidance on how to correctly export and compile the
    Whisper-base encoder ONNX for DFC 3.28.x?

Hardware: Hailo-8 (not 8L), amd64
HailoRT version: 4.19.0
DFC version: 3.28.0
Whisper variant: base

Thank you

2

Hi @user705,

Welcome to the Hailo Community!
In order to convert the Whisper-base model for Hailo-8 using the hailo-whisper, you need at least version 3.31.0 of the Hailo DataFlow Compiler. That version includes some changes in the parser/compiler that are necessary to successfully convert the Whisper tiny/base models.
Of course, you would need a more recent version of HailoRT, too, which is not possible in your case.

We got an indication that Bosch Rexroth is planning to upgrade the HailoRT version as part of a future CtrlX OS update. If you are in touch with Bosch Rexroth’s team, we would recommend checking with them about the scheduled timeline for the update.

3

Thank you @pierrem!

I will definitely get in touch with the Bosch Rexroth team to inquire about their timeline for the HailoRT update in CtrlX OS.

Thanks again for your support!