FAQ

Does this package pick a provider automatically, or route/fall back between providers?

No. __init__.py exports a __llm_providers__ list of provider name strings, but nothing in the package reads that list to select a class dynamically — you import and instantiate the specific interface class you want (OpenAIInterface, AnthropicInterface, etc.) directly. There is no automatic failover if one provider errors.

What is __hugginface_models__ (in __init__.py) and hugginface_model (in llm_config) for?

They're a list/field of Hugging Face model-family name strings (gpt, llama, mistral, bert, arabert, etc.) and a default Hugging Face model id ("EleutherAI/gpt-neo-125M"), respectively. Note both carry a spelling of "hugginface" (missing the second g in "Hugging"). Neither is read or used by any interface class in this package — there is no HuggingFaceInterface implemented here. They appear to be reserved for a provider that doesn't yet exist in this codebase.

Can I use one interface instance across multiple threads?

Not something the package documents or guarantees. Thread safety depends entirely on the underlying provider SDK's client (or, for Ollama, on requests/aiohttp usage patterns) — this wrapper adds no locking of its own. See each interface's "Thread safety" note on its API page.

Do these interfaces track token usage or cost?

No — none of the generate/generate_async/astream methods read or return token-usage/cost fields from the underlying SDK responses (e.g. OpenAI's usage field, Anthropic's usage field). Only the generated text is returned.

Why do some providers return a dict on error and others return a str?

It's a genuine inconsistency across the modules, not an intentional design split by provider "tier." OpenAI, Anthropic, Groq, and AWS Bedrock return {"error": str(e)}; Gemini, Mistral, and Ollama return plain error-prefixed strings. See the Provider Comparison table and check accordingly in your own code, or use validate_connection() for a structured result. Note this inconsistency is specifically about the error path — the tool-call success path is uniform across all seven: a successful tool call always returns the same {"content", "tool_calls", "stop_reason"} dict shape, even on Gemini/Mistral/Ollama, so isinstance(result, dict) alone is no longer enough to conclude "this failed" once you're passing tools — check for "error" vs "tool_calls" as the key. See Tool Calling.

Is there a way to list which providers/models are available without instantiating every interface?

Not built in. __llm_providers__ gives you the list of provider names the package supports, but there's no unified "list available models for provider X" call across interfaces — the closest is OllamaInterface.list_models(), which is specific to a local Ollama server and has no equivalent on the hosted-provider interfaces.

Does astream() always give me lower latency to first token?

Only for OpenAI, Anthropic, Groq, and AWS Bedrock (genuine incremental streaming), and for Mistral when its native stream call succeeds. For Gemini, and for Mistral/Ollama on their fallback paths, astream() still waits for the entire response internally before yielding anything — see BaseLLMInterface.astream.

Do all seven interfaces support tool/function calling, and is the schema the same everywhere?

Yes to both. generate()/generate_async() on every interface accept a tools argument (and optional tool_choice) in one unified schema, converted internally to each provider's native format by llm/tools.py. If the model calls a tool, the return value is a dict — {"content", "tool_calls", "stop_reason"} — instead of a plain string, on every provider (including the three that otherwise represent errors as strings, not dicts). See Tool Calling for the full contract.

Does astream() support tool calling / streamed tool-call deltas?

No, on any of the seven interfaces. astream() only ever yields text; it doesn't accept tools as a named parameter, and none of the seven implementations reconstruct tool-call information from a stream. If you need tool calls, use generate() or generate_async() for that turn instead of astream().

Does this package execute my tool/function for me, or manage the multi-turn tool-result loop?

No. It converts your tool schema to each provider's format and parses tool-call responses back into a unified shape — that's the extent of it. Running the tool and sending its result back (as a follow-up messages entry, in whatever shape that provider's SDK expects for a tool-result turn) is on you. See Tool Calling § Sending the tool's result back.