Hi
I’ve created a service for my Raspberry Pi that, based on Hailo 10h, provides various services and models through different interfaces.
It supports various protocols, including those from OpenAI.
I’ve also integrated two virtual assistants for Home Assistant and Frigate so you can use the service there. Various requests are processed deterministically and do not go to the LLM/VLM.
There’s a status GUI that lets you configure and monitor the system, as well as a playground to test the different models.
I have it running directly on the Pi. However, you could also run it in Docker or Proxmox/LXC.
I welcome feedback and ideas for improvement