Hailo Ollama on Raspberry PI 5 AI Hat+ 2Embedding Models for RAG

General Discussion
1

Hi everyone,

does the ai hat+ 2 support some embedding models to encode text for RAG applications and vector databases like chromadb?

It seems in the currently available repository there is no embedding models which means RAG applications are currently not possible?
Is there a guide how i could convert existing embeddings models into hef files?

2

Hi,

While HAT+2 might not be the optimal solution for DB scale generation, we have the CLIP implementation which is the closest:

In addition we have example with Vector DB usage - which might be a helpful reference:

Thanks,

3

Hmm i was more thinking like converting a embedding modell (e.g. sentence-transformers/all-MiniLM-L6-v2) via Dataflow Compiler into a .har and then a .hef file.

This way i thought i could get an LLM typical embedding model.
But the process seems not to work.
I was able to create a .har file but i’m stuck at quantization process step before converting to a .hef file.

4

once i have the .hef file i can run the embedding processes in the RAG pipeline on the pi5 with the hat2+

At least that is my thinking process?

5

Hi,

Can you please elaborate on “stuck at quantization process step before converting to a .hef file”?

Thanks,