I created python GUI for a 2 way voice chatbot on a Rpi 5 using whisper for the speech to text. Operation is simple. Select a LLM you are going to be prompting by a dropdown list of available LLMs such as Qwen2.5-1.5B. Whisper-Base is used for the speech to text, and piper for text to speech for the response from what ever LLM you selected . Seeing you can’t have 2 models running at the same time you have to switch between the two. This pretty much works perfectly using a simple button in the GUI to start recording your verbal prompt. You also have the option to just type in your prompt as text instead. The Whisper gets loaded fine, and after it’s speech-to-text processing is completed Whisper gets unloaded(released) and Qwen2.5-1.5B gets loaded and gives the response back through piper. The problem at times, but not always is releasing Qwen2.5-1.5B back to whisper. I usually get an error that the model is not releasing, or is taking way to long to release. I’m just looking to see if anyone has attempted creating a project like this and had issues with loading and unloading of the models. I actually had it working good, but decided to make this work like google’s alexa using a wake word. It detects the wake word fine but issues started when I put the whole process in a loop to continually check for “wake” words and process with whisper and then back to Qwen2.5-1.5B using piper for output. The whole process then loops to start over and waits for a wake word again. Switching of the LLMs are causing problems.
1 Like
Hi @BryanSwie ,
Maybe this example can be helpful - It points to a specific line in the code: hailo-apps/hailo_apps/python/gen_ai_apps/simple_whisper_chat/simple_whisper_chat.py at main · hailo-ai/hailo-apps · GitHub
Thanks,