Using hailo-ollama server with ollama client

[AlbertaBeef]

Testing out the Hailo-10 on the RPI5.

The “hailo-ollama” API is working.

My RPI5+Hailo-10 is at IP address 10.0.0.108, so the following is working from a separate computer:

curl --silent ``http://10.0.0.108:8000/api/chat`` -H ‘Content-Type: application/json’ -d ‘{“model”: “qwen2:1.5b”, “messages”: [{“role”: “user”, “content”: “Who are you?”}]}’

Is this API compatible with the OLLAMA API ?

I tried the following, which is not working:

export OLLAMA_HOST=10.0.0.108:8000

ollama run qwen2:1.5b “Who are you?”

I get the following error from ollama:

Error: something went wrong, please see the ollama server logs for details

Before I debug, I wanted to check if this can be expected to work.

Thank you in advance for any feedback or guidance,

Mario.

[Nadav]

Hi @AlbertaBeef,

Very good question, before I go to the answer, I want to highlight the ability to get immediately through our AI assistant on the Deverloper-Zone:

I’ve just pasted your full question there to get the immediate answer-

Using ollama run ... against hailo-ollama is not expected to work, even if you set OLLAMA_HOST, because the Hailo server only implements the REST side, not the full Ollama CLI protocol. This is something that is on our working plans.

[AlbertaBeef]

@Nadav, wow ! That is a very cool feature !
And thank you for the response