add also unsupported models to hailo_model_zoo_genai for hailo-ollama

General Discussion
,
1

Hi there,
i installed and used GitHub - hailo-ai/hailo_model_zoo_genai: Model zoo for Gen AI models for Hailo products at my raspberry pi 5.
would it be possible in future to also let the user download more even unsupported models like 2B Models or other models which run then as fallback on the PI instead of the hailo accelerator?
what also would be great is if there is some doc how users can train and add more models to hailo_model_zoo_genai/models/manifests at main · hailo-ai/hailo_model_zoo_genai · GitHub

2

Hi,

We’ve recently updated our retraining guide: hailo-apps/doc/developer_guide/retraining_example.md at main · hailo-ai/hailo-apps · GitHub

Regarding mixing supported and unsupported - maybe Open WebUI (or similar) could be a solution here by connecting it to multiple local Ollama servers?

Thanks,

1 Like
add also unsupported models to hailo_model_zoo_genai for hailo-ollama
3

hi @michael connecting multiple ollama Servers to one UI won’t Work, cause ollama does round robin when you have more then one backend, so sadly this is no option.

thanks, but in this case there is documentation needed how to train ll Models, there are even some 2B Models Out there Like gemma2 or gemma3 which could also work with 8gb vram, or also some Models with visual detection Support, thats why i ask because Main usage focus for me is the llm section where more Models are required.

4

Hi,

@user405

Regarding fallback use case:
I tested the following and it worked:

  1. Regular ollama on http://127.0.0.1:11434

  2. Hailo ollama on http://localhost:8000

Then both can be accessed, for example from Open WebUI running on http://localhost:8080 by adding both as local connections, or via API and then it’s possible to program a fallback use case to switch automatically.

Regarding training GenAI models:
We are planning in the future to release support for LoRA fine tuning.

Thanks,

1 Like
6

@michael is there also a plan of hailo to release more models which are out of the box supported? such like deepseek-coder or gemma2:latest

7

Hi,

We do plan to enable more models, however for time being we can’t share details and timelines.

Thanks,