Whisper + Hailo8(L) + FastAPI

Community Projects
, ,
1

Hello everyone!

I’d like to share my modification of the speech_recognition project from Hailo-Application-Code-Examples.

With this update, you can easily deploy a Docker container on your Raspberry Pi 5 that accepts audio files via an API and returns transcribed text as a response.

I started this project to integrate speech recognition into my smart home system and a custom Telegram chat bot. I’m planning to add new features and improvements soon—so stay tuned!

As of June 26, this is the first working version.

Tested on Ubuntu Server 25.04, Hailo-8L, HailoRT v4.21, Python 3.11.9.
Docker container works on Ubuntu 24.04, Python 3.11.9.

I’d be happy to hear your questions, suggestions, and feedback!

4 Likes
2

Great achievement @MafiaCoconut! Can’t wait to test this!

3

Congrats! I am trying to do this 100% on the Pi and keep running into issues of what to put in the python code to get Hailo started, I have all the files and I got the demo to work, but I want to be able to pass my own wav to it from the Pi. I am complete noob, so any help would be appreciated!

4

Hi, i am going to build all my home system on services with API (whisper, LLM, piper) so you can follow me on GitHub and wait for my new releases regarding that)

4 Likes
5

Important update!

Now with the version v.1.0.0 this service is also working with Wyoming Protocol for HA voice assistant

A TCP server has been added to the FastApi server. Accessing Hailo is activated only when a file is sent to the service for processing, so the Hailo-8(L) can process other tasks in parallel.

2 Likes
6

Hi @MafiaCoconut ,

I have been able to get it working on 4.20. How do I use it with homeassistant. At the moment i have only successfully tested sending a .wav file for STT.

Thanx in advance

7

Hi, you need to open Wyoming Protocol on your home assistant, than in ip write ip address of your device in local network and in port 10300

8

Hi @MafiaCoconut .
I appreciate the response.
I will keep you posted.

9

Hello again @MafiaCoconut.
I have included whisper using Hailo-8L in Home Assistant successfully. I also observe the encoder and decoder process using HAILO_MONITOR. Everything works splendidly. I have replaced the whisper add-on provided by HA with the whisper using the Hailo-8L service on my container. I re-used a container with Hailo-Examples and did not have to build a new container for the service. I just created a new ’ .venv’ Python environment and included all the requirements you provided. As I mentioned earlier, I tried it using HailoRT v4.20 and all the dependencies and drivers.

10

Hi @Nasiru_Aboki , I’m glad that everything is working for you!)

1 Like
11

Benchmarks

Hey @MafiaCoconut , many thanks for your work!
Do you have any sort of Benchmarks for this on your setup?

I’m especially wondering how it compares to whisper running on RPi 5 without the HAILO8.
Because the performance & accuracy seen in this video is really not enough for my purpose:

Thanks in advance!

12

Hi @Susanne_Schneider,

I have no benchmarks for my case, but i’ve tested it with my homeassistnat build and HAILO-8L(13 TOPS) and for sentence “turn off light in the hallway” was trancribed in

without HAILO-8L - 8-10 seconds
with HAILO-8L - 3-4 seconds

It works fine for home voice assistant, but i am not sure if it will work good enough for another ways

Also maiby with the bigger version HAILO-8 this will improve the speed of text recognition to 1-2 seconds, but unfortunately I can’t test it because my RP5 doesn’t support the AI hat.