OllamaInterface

llm.OllamaInterface — interface to a local Ollama server, over its HTTP API. The only interface in this package that doesn't wrap a hosted-provider SDK, and the only one that can manage a background server process on the caller's behalf.

Constructor

OllamaInterface(
    model_name: str = config.ollama_model,   # "llama2"
    api_key: Optional[str] = None,
    **kwargs,
)

api_key is accepted for interface consistency but is not used — Ollama's local HTTP API has no authentication in this implementation.

Extra settings read from **kwargs (not from the standard constructor parameter list — pass them as keyword arguments):

Server lifecycle at construction time

_start_server() launches ollama serve as a detached background process — using CREATE_NO_WINDOW | DETACHED_PROCESS flags on Windows, start_new_session=True on other platforms — and polls /api/version every 0.5 seconds up to server_start_wait seconds. If the ollama binary isn't on PATH, this raises internally (FileNotFoundError), which is caught and logged as a warning ("❌ Ollama not found. Install from: https://ollama.com/download") — it does not raise out of __init__.

Important: self._we_started_server is only set True if this instance launched the process. stop_server() (and the destructor, below) will never terminate a pre-existing Ollama server that this instance merely connected to.

Message building: _build_messages(prompt, messages)

@staticmethod
def _build_messages(prompt, messages):
    """Resolve prompt / messages into a messages list (mirrors OpenAI interface)."""
    if messages is not None:
        return messages
    if prompt is not None:
        return [{"role": "user", "content": prompt}]
    raise ValueError("Either 'prompt' or 'messages' must be provided.")

Same shape as OpenAI/Anthropic/Groq/AWS/Mistral's helper of the same name — no role translation needed, since Ollama's /api/chat endpoint accepts the same flat OpenAI-style {"role": ..., "content": "..."} dicts. Unlike every other provider, this helper is only called when messages is not None — see the endpoint-switching note below for why that matters to the ValueError behavior.

generate(messages=None, prompt=None, max_tokens=2048, temperature=0.3, tools=None, tool_choice=None, **kwargs) -> Union[str, dict]

Posts to {base_url}/api/generate with "stream": True internally — even for the synchronous, non-streaming generate() method — and accumulates the full response before returning it. Per the source docstring, this is deliberate: streaming the HTTP response internally means each received token resets the read-timeout clock, so long generations don't time out mid-response the way a single non-streaming HTTP call with a fixed read timeout might. This internal-streaming behavior is bypassed entirely when tools is set — see the dedicated section below.

Which endpoint gets called depends on whether you pass messages:

if messages is not None:
    msgs = self._build_messages(prompt, messages)
    url = f"{self.base_url}/api/chat"
    payload = {"model": self.model_name, "messages": msgs, "stream": True, "options": options}
else:
    if prompt is None:
        raise ValueError("Either 'prompt' or 'messages' must be provided.")
    url = f"{self.base_url}/api/generate"
    payload = {"model": self.model_name, "prompt": prompt, "stream": True, "options": options}

OllamaInterface is the only provider in this package that switches HTTP endpoint based on messages vs. prompt, rather than just reshaping the same payload — /api/chat when messages is given, /api/generate when only prompt is given. The streamed response shape differs accordingly: each line is parsed as chunk["message"]["content"] on the /api/chat path vs. chunk["response"] on the /api/generate path. This endpoint choice only applies when tools is not set — see below.

Parameters

Uses timeout=(10, config.time_out) — a 10-second connect timeout and a config.time_out (200s) read timeout that resets per-chunk due to the internal streaming (the tools path below uses this same (10, config.time_out) tuple, but as a single fixed non-streaming read timeout rather than one that resets per chunk).

Returns: If tools is not set, the concatenated response string (str) on success. On connection failure: "❌ Error: Cannot connect to Ollama server. Make sure it's running.". On timeout: "⏱️ Error: Request timeout. Try increasing timeout value.". On any other exception: f"⚠️ Warning in OllamaInterface: {str(e)}" — these error paths return a string, never raise, same pattern as Gemini's async path and Mistral. If tools is set and the model returns tool calls, this returns the unified {"content", "tool_calls", "stop_reason"} dict instead — see below.

ValueError is the one exception to the string-error pattern, and it's specific to this class: if messages is None and tools is not set, generate checks prompt is None before building any payload and raises ValueError("Either 'prompt' or 'messages' must be provided.") directly. If tools is set, the same error condition is instead raised by self._build_messages(prompt, messages) — the same helper OpenAI/Anthropic/Groq/AWS/Mistral use. Either way, this class has a dedicated except ValueError: raise ahead of the generic except Exception handler, so the ValueError always propagates straight to the caller uncaught in both the tools and no-tools paths — the same as GeminiInterface.generate's behavior on the sync path.

Tool calling forces a non-streaming /api/chat request

if tools:
    msgs = self._build_messages(prompt, messages)
    url = f"{self.base_url}/api/chat"
    payload = {
        "model": self.model_name,
        "messages": msgs,
        "stream": False,
        "options": options,
        "tools": to_ollama_tools(tools),
    }
    response = requests.post(url, json=payload, timeout=(10, config.time_out))
    ...
    data = response.json()
    message = data.get("message", {}) or {}
    calls = parse_ollama_tool_calls(message)
    if calls:
        return {"content": message.get("content") or None, "tool_calls": tool_calls_to_dicts(calls), "stop_reason": data.get("done_reason")}
    return message.get("content", "")

As soon as tools is truthy, generate takes an entirely different path from everything described above:

1. It always uses /api/chat (building messages from prompt via _build_messages if you only passed prompt) — the prompt → /api/generate branch is skipped entirely, regardless of whether messages was given.
2. It sends "stream": False and issues a single non-streaming requests.post(...), rather than the internal per-chunk streaming used everywhere else in this class — because Ollama only reports tool_calls on the final, complete message, so there's nothing to gain from streaming here, and using requests.post(..., timeout=(10, config.time_out)) as a fixed (not per-chunk-reset) timeout is an accepted trade-off specifically for this path.
3. tools is converted via llm.tools.to_ollama_tools — a plain alias of to_openai_tools, since Ollama's tool-calling dialect matches OpenAI's — and merged into the request body's top-level "tools" key.
4. The response is parsed as a single JSON object (not NDJSON lines): message.get("tool_calls", []) is read by parse_ollama_tool_calls, which parses each call's arguments from a JSON string if the model returned one as a string, or uses it as-is if it's already a dict. If any calls are found, the unified {"content", "tool_calls", "stop_reason"} dict is returned (stop_reason comes from Ollama's own done_reason field). If no calls are found — e.g. the model or a smaller/older model on the server doesn't support tool calling and just answers in prose — the plain message.get("content", "") string is returned instead, same as the no-tools path.

tool_choice is accepted as a parameter for interface symmetry with the other providers but is not forwarded anywhere in this branch — Ollama's tool-calling API has no equivalent concept, so passing it has no effect.

Example

from zerda.llm import OllamaInterface

llm = OllamaInterface(model_name="llama2")  # auto-starts local server if needed
print(llm.generate(prompt="اشرح مفهوم embeddings ببساطة."))

# multi-turn — posts to /api/chat instead of /api/generate
print(llm.generate(messages=[
    {"role": "user", "content": "What is RAG?"},
]))

# tool calling — forces /api/chat with stream=False regardless of messages/prompt
tools = [{"name": "get_weather", "description": "...", "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}}}]
result = llm.generate(prompt="Weather in Cairo?", tools=tools)

generate_async(messages=None, prompt=None, max_tokens=2048, temperature=0.3, tools=None, tool_choice=None, **kwargs) -> Union[str, dict]

Same endpoint-switching and _build_messages logic as generate above, over aiohttp if available, streaming the response the same way as the sync path. If aiohttp is not installed, this method does not raise for that reason — it falls back to running the synchronous generate() in a thread via asyncio.to_thread(...), now passing tools/tool_choice through positionally along with the rest of the arguments, so generate_async still works without aiohttp (just without a native async HTTP client underneath) including on the tools path. It carries the same ValueError-propagates-uncaught behavior as generate for a missing prompt/messages on the no-tools path (via the same except ValueError: raise guard), and the same tools path described above — built with aiohttp.ClientSession().post(...) and aiohttp.ClientTimeout(connect=10, sock_read=config.time_out) instead of requests.post, but otherwise identical: "stream": False, to_ollama_tools, parse_ollama_tool_calls, and the same unified return dict.

astream(messages=None, prompt=None, max_tokens=2048, temperature=0.3, **kwargs) -> AsyncIterator[str]

With aiohttp available: yields each token from the streamed response as it arrives — genuine incremental streaming, switching between /api/chat and /api/generate the same way as generate. Without aiohttp: falls back to running the sync generate() in a thread and then yielding the result word-by-word (same fallback pattern used elsewhere in this package).

astream does not accept tools/tool_choice as named parameters and does not implement the tools branch described above — it only ever calls the streaming/no-tools code paths. Use generate/generate_async for tool calling. See Tool Calling § Streaming limitation.

[!WARNING] astream()'s ValueError behavior depends on whether aiohttp is installed This is a genuine quirk, verified against the source — astream() does not have the same except ValueError: raise guard that generate/generate_async have:

• With aiohttp installed: a missing prompt/messages raises ValueError inside the main try block, which is caught by the generic except Exception as e: yield f"⚠️ Ollama stream error: {e}" — so you get a yielded error string, not a raised exception.
• Without aiohttp installed: the ImportError fallback branch calls self.generate(messages, prompt, max_tokens, temperature, **kwargs) directly, outside any further try/except of its own — and since generate() does raise ValueError uncaught for a missing prompt/messages, that exception propagates straight out of astream() too in this case.

In short: whether iterating astream() with no prompt/messages raises or just yields an error string depends on an installed-package detail unrelated to your call itself. If you need deterministic behavior here, validate prompt/messages yourself before calling astream().

Model management methods

Unique to this interface — no other provider class exposes local-model management, since only Ollama runs models locally:

list_models() -> list

GET {base_url}/api/tags, returning the name field of each entry in the response's models list. Returns [] on any error (logged via print, not raised).

pull_model(model_name: str = None) -> bool

POST {base_url}/api/pull with {"name": model or self.model_name}, timeout=600. Returns True/False based on HTTP status; prints progress messages ("📥 Downloading model: ...") rather than using logging.

Server shutdown

stop_server()

def stop_server(self):
    if self.server_process and self._we_started_server:
        ...

Only acts if this instance actually launched the process (_we_started_server). Attempts a graceful terminate() with a 5-second wait(); falls back to kill() if that raises.

__del__

def __del__(self):
    self.stop_server()

Calls stop_server() on garbage collection — combined with the guard above, this means an OllamaInterface instance will clean up an Ollama process it started, but will never kill a server that was already running before the instance was created.

[!WARNING] __del__ timing is not guaranteed Python does not guarantee __del__ runs promptly (or at all, e.g. in the presence of reference cycles or at interpreter shutdown). Don't rely on this as your only mechanism for stopping a server you started — call stop_server() explicitly when you're done if reliable cleanup matters.

Related APIs

• BaseLLMInterface
• Configuration — ollama_model, ollama_base_url, time_out all trace to llm_config here.
• Tool Calling — the unified tools/tool_choice contract; Ollama is the one provider whose request shape (endpoint + streaming) changes specifically because tools was passed.