I think this may be going in a similar direction as
but since I am not sure I am opening the issue with a more precise description.
The goal is to use H10 as LLM provider for agentic coding using OpenCode.
Since hailo-ollama implements only the REST API and not the full functionality of Ollama
and it is thus not possible to connect OpenCode to it I want to use Open WebUI as intermediary.
Using v5.3.0 in conjunction with OWUI 0.9.5 works, but requires a fix of user input sanitisation
to prevent JSON parsing errors (JSON config - HAILO_INTERNAL_FAILURE(8)).
With the fix I observe no problems interfacing with the LLMs through the web interface.
Now for the problem:
When attempting to submit a simple query (e.g. Write a Hello World script in Python.) from OpenCode via OWUI
I always encounter Server Connection error.
Looking at OWUI debug output I see
Failed to parse error response: 500, message='Attempt to decode JSON with unexpected mimetype: ', url='http://localhost:8000/api/chat'
This indicates that whatever is returned by hailo-ollama to OWUI is not formatted as expected.
This is likely a hailo-ollama-specific issue, because trying the same with an Ollama docker container (on a different system)
works.
My question is thus: Is this issue known and is there a (simple) fix for it?
I originally thought the issue was the use of application/x-ndjson as return type rather than plain application/json, but switching did not help, so thatâs probably not the solution
Since youâve already resolved the newline sanitization issue and direct OWUI chat works, the remaining problem likely comes down to OpenCode (through OWUI) sending Ollama API request fields or parameters that hailo-ollama doesnât handle or different streaming expectations - which causes hailo-ollama to hit an error path and return a 500 without proper JSON content-type headers, hence the âunexpected mimetypeâ on OWUIâs side.
You might try inspecting the exact request body that OWUI forwards to hailo-ollama when the error occurs (e.g., by running hailo-ollama with verbose logging or placing a simple logging proxy between OWUI and hailo-ollama on port 8000) to identify which specific field or request shape triggers the failure - once you know that, you could strip or modify those unsupported fields in an OWUI pipe/filter function before the request reaches hailo-ollama.
Identifying the exact triggering field from your logs would be very helpful if youâd like to share it here.
Hi!
By the ancient art of printf-debugging I have previously managed to
narrow down the origin of the error to the non-streaming section of handle_completion. But that appears to be a red herring.
The server error is indicated in OpenCode immediately after submitting the query. Looking at the same time at the output from hailo-ollama it takes it a while to process the prompts but it appears to get all of them. Which currently suggests that the error may be actually due to the way the request is processed by oatpp.
Since the current implementation uses oatpp::network::Server rather than ouatpp::web::server
it doesnât seem like I can register request/response interceptors to get in-depth info on what is
being received where (or at least itâll take a bit more work). For now Iâve dumped the traffic on the hailo-ollama port using ngrep and will have a closer look at that.
The error Iâm getting through the capture at run time is:
server=oatpp/1.4.0
code=500
description=Internal Server Error
stacktrace:
- [oatpp::data::mapping::TreeToObjectMapper::mapString()]: Node is NOT a STRING
- [ApiController]: Error processing request
- Error processing request
Iâll get back to you once Iâve had a more in-depth analysis of the received data.
It seems like I was able to somewhat pinpoint the source of the errors: Function calling. Specifically: The JSON request containing a tools array.
I was able to elicit the same error when querying qwen3:1.7b from OWUI. Disabling Reasoning and Stream Chat Response in the controls of the chat and setting Function Calling to Native instead of Default will result in this error.
The packet sent prior to the error consists of a JSON object containing the members model, messages, options, stream, tools, where
options just contains num_ctx:null and temperature:0.7
stream contains stream:false
tools contains an array of objects describing available functions, their parameters etc.
This packet consistently results in the mapping error and thus the server error.
In contrast, turning Function calling to Default instead will result in a packet consisting of all of the above members excepttools and will succeed.
How to fix
A provisional fix for OWUI is to disable Builtin Tools as one of the Capabilities of the model in OWUI admin settings of the model. This will disable the tools field even when using the Native option.
This will not fix the issue for external tools that use OWUI as a provider (e.g., OpenCode). They can pass their own set of tools and trigger this problem. To fix the problem also for external tools a filter for the model must be implemented that strips the tools member from the message body.
The simplest way is to go to admin settings>Functions and create a new function with the only action in the inlet being body.pop("tools"):
from pydantic import BaseModel, Field
from typing import Optional
class Filter:
class Valves(BaseModel):
pass
def __init__(self):
self.valves = self.Valves()
pass
async def inlet(self, body: dict, __user__: Optional[dict] = None) -> dict:
# Remove the tools
body.pop("tools")
return body
async def outlet(self, body: dict, __user__: Optional[dict] = None) -> dict:
# We do not have to modify what the model returns to us.
return body
The function must then be enabled and also enabled under âFiltersâ for each model you want to use it for. In my case I enabled it only for qwen2.5-coder:1.5b so as to avoid breaking other models.
With the filter in place I can now invoke OpenCode without a server connection error.