Tool Calling

All seven interfaces accept a tools argument (and an optional tool_choice) on generate() and generate_async(). The schema you write is identical across providers; the llm/tools.py module converts it to each provider's native shape and normalizes each provider's response back to one shape.

[!NOTE] Not covered here: streaming tool calls astream() does not reconstruct tool-call deltas on any provider — it only ever yields plain text. Use generate() / generate_async() when you need tool calls; see Streaming limitation below.

The unified tool schema

tools = [
    {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "input_schema": {                # alias: "parameters" (either key works)
            "type": "object",
            "properties": {
                "city": {"type": "string"},
            },
            "required": ["city"],
        },
    },
]

Every tool needs a name. description defaults to "" if omitted. The JSON-schema object can be given under either input_schema (Anthropic-style) or parameters (OpenAI-style) — llm/tools.py's internal _normalize_tool() accepts both, so you don't need to know which dialect a given provider prefers.

Calling with tools

from zerda.llm import AnthropicInterface

llm = AnthropicInterface(api_key="sk-ant-...")
result = llm.generate(prompt="What's the weather in Cairo?", tools=tools)

tool_choice is optional and forwarded to the provider mostly as-is; its accepted values are provider-specific (see the table below).

The unified response shape

If the model does not call a tool, the return value is a plain string (or, on OpenAI/Anthropic/Groq/AWS Bedrock's error path, the same {"error": ...} dict as always) — identical to calling without tools at all. Existing code that doesn't pass tools is completely unaffected.

If the model calls one or more tools, the return value becomes a dict:

{
    "content": "...",       # any text the model produced alongside the call, or None
    "tool_calls": [
        {"id": "call_abc123", "name": "get_weather", "arguments": {"city": "Cairo"}},
        # ... one entry per tool call
    ],
    "stop_reason": "...",   # the provider's own finish/stop-reason value, unmodified
}

arguments is always a dict — each interface parses whatever the provider actually returns (a JSON string for OpenAI/Groq/Mistral, an already-parsed object for Anthropic/Bedrock/Gemini) before handing it back to you, so you never need provider-specific parsing code. id is None for Gemini and for Ollama tool calls that don't include one — Gemini's Converse-equivalent function-call parts and Ollama's tool_calls don't always carry a call id.

[!WARNING] Check isinstance(result, dict) before assuming plain text Because a successful tool call and a plain-text answer share the same method and the same success path, calling code that only ever expects a string must now check for a dict even outside the existing error-handling path (see Provider Comparison for each provider's error-dict shape, which is a separate case from the tool-call dict shape described here — both are dicts, but with different keys, so check for "tool_calls" vs "error" if you need to tell them apart).

Sending the tool's result back

This package converts schemas and parses responses — it does not execute your function or manage the follow-up turn for you. After you run the tool yourself, append the result to messages in whatever shape that specific provider's SDK expects for a tool-result turn (this is provider-native, not unified by this package) and call generate/generate_async again. Consult the specific SDK's docs for the exact tool-result message shape per provider (e.g. an assistant message with tool_calls followed by a tool role message for OpenAI-style APIs, or a user message containing a tool_result content block for Anthropic).

Per-provider conversion

Ollama forces a non-streaming request

OllamaInterface.generate/generate_async normally stream internally (to reset Ollama's read-timeout per chunk) and hit /api/generate or /api/chat depending on whether you passed prompt or messages. As soon as tools is set, both methods instead:

1. Build a messages list from prompt if you didn't already pass one.
2. POST to /api/chat with "stream": False — Ollama reports tool_calls on the final, complete message, so a non-streaming request is used specifically for the tools path (this is a genuine special case; every other provider keeps its normal request shape when tools is passed).
3. Parse tool_calls off the single JSON response instead of iterating NDJSON lines.

If your Ollama server/model doesn't support tool calling, expect a plain response with no tool_calls key (Ollama omits the key rather than returning it empty), which this package correctly treats as a normal plain-text result.

Streaming limitation

None of the seven astream() implementations reconstruct tool-call deltas — even on OpenAI, Anthropic, Groq, and AWS Bedrock, which stream real incremental text. Passing tools through to astream()'s **kwargs (where the underlying SDK accepts it) may still influence whether the model attempts a tool call, but the streamed output only ever surfaces text tokens; any tool-call information in the stream is not parsed or returned. If you need tool calls, call generate() or generate_async() instead — accept the full round-trip instead of a stream for that turn.

Related pages

• Quickstart § Tool calling for a minimal end-to-end example.
• Provider Comparison for how tool calling fits alongside each provider's other behavioral differences.
• llm.tools API reference for the exact converter/parser function signatures.
• Any provider's API page under API Reference for that provider's exact tools/tool_choice parameter notes.