Provider Comparison

On this page

Choosing an interface
Related pages

A single-glance summary of behavioral differences across the seven interfaces. Every row below is drawn directly from the source of each module — see the linked API page for details and code.

	OpenAI	Anthropic	Groq	Gemini	Mistral	Ollama
Accepts `messages` list	✅	✅	✅	✅ (translated: `assistant`→`model`, `system`→separate `system_instruction`)	✅ (passed straight through)	✅ (switches `/api/generate` → `/api/chat`)
Accepts `tools` (unified schema)	✅	✅	✅	✅	✅	✅
Tool call `id` present	✅	✅	✅	❌ (always `None`)	✅	Only if the model/server returns one, else `None`
Tool calling forces non-streaming request	❌	❌	❌	❌	❌	✅ (switches to `stream: False` only when `tools` is set)
Native incremental streaming	✅	✅	✅	❌ (word-split fallback)	✅ (falls back to word-split on error)	✅ with `aiohttp`; word-split fallback without
General error return shape (API/network failures)	`dict` (`{"error": ...}`)	`dict`	`dict`	`str` (`"Error: ..."`)	`str`	`str`
Missing `prompt`/`messages` raises `ValueError` uncaught?	❌ (caught → `dict`)	❌ (caught → `dict`)	❌ (caught → `dict`)	`generate()`: ✅ raises. `generate_async()`/`astream()`: ❌ caught → `str`	❌ (caught → `str`, all three methods)	`generate()`/`generate_async()`: ✅ raises. `astream()`: ❌ if `aiohttp` installed (yields `str`); ✅ if not (fallback re-raises)
Sync + async clients created eagerly	✅ (both)	✅ (both)	✅ (both)	N/A (single client, sync calls run via executor for async)	N/A (single client for both)	N/A (plain HTTP, no SDK client)
Has `cleanup()` (sync)	✅	✅	✅	❌	❌	❌ (uses `stop_server()` instead)
Has `acleanup()` (async)	✅	✅	✅	❌ (inherits base no-op)	✅	❌ (inherits base no-op)
Built-in retry/backoff	❌	❌	❌	✅ (5 attempts, exponential + jitter, on 429/503/UNAVAILABLE)	❌	❌
Built-in rate limiting	❌	❌	❌	✅ (hardcoded 5s min gap between calls)	❌	❌
Reads default model from `llm_config`	N/A (`gpt-4` literal)	N/A (literal)	❌ (literal, matches `config.groq_model` value)	✅ (`config.gemini_model`)	✅ (`config.mistral_model`)	✅ (`config.ollama_model`)
Local process management	❌	❌	❌	❌	❌	✅ (`auto_start`, `stop_server()`, `__del__`)
Model management helpers (`list_models`, `pull_model`)	❌	❌	❌	❌	❌	✅
Logging style on errors	`logging` module	`logging` module	`logging` module	`print()`	`print()` (sync path only)	`print()`

Choosing an interface

Need multi-turn message-history calls? All seven interfaces now accept messages directly — see Architecture § prompt vs messages for the payload-shape differences per provider (Gemini's role translation, Ollama's endpoint switch, AWS's block-shape requirement).
Need tool/function calling? All seven interfaces accept the same unified tools schema — see Tool Calling for the per-provider conversion table and the one caveat (Ollama forces a non-streaming request when tools is set).
Calling many models through one AWS account/IAM setup, including non-Anthropic models? AWS Bedrock via AWSInterface lets you swap model_name (e.g. Claude, Llama, Titan, Mistral, Nova) without changing code.
Running fully offline/local, no API key? OllamaInterface — but budget for its extra startup behavior (auto-starting ollama serve, or reachability checks) at construction time.
Working against Gemini's free tier? GeminiInterface's built-in rate limiting and retry logic exist specifically for this — but be aware every call can add up to ~5 seconds of enforced spacing.
Need the error path to be easy to detect programmatically? OpenAI/Anthropic/Groq/AWS return a dict with an "error" key on failure; Gemini/Mistral/Ollama return a plain "Error: ..."-prefixed string, so you'd check with result.startswith("Error")/"❌" instead of isinstance(result, dict) — but see the ValueError-uncaught row above before assuming every failure mode on Gemini/Ollama comes back as a string rather than an exception.

Architecture for the shared contract every interface follows.
Tool Calling for the unified tools schema and per-provider conversion.
Configuration for exactly which llm_config fields are (and aren't) wired up per provider.