ZerdaZerda

Configuration

On this page

Field reference

FieldTypeDefaultUsed by
max_tokenint2048All providers, as the default max_tokens in generate/generate_async/astream.
temperaturefloat0.3All providers, as the default temperature.
top_pfloat0.9Gemini (default_config); Ollama (generate/generate_async request options, via kwargs.get("top_p", 0.9) — note this literal default is hardcoded in Ollama's methods rather than read from config.top_p).
top_kint50Gemini (default_config). Ollama uses its own hardcoded 40 for top_k rather than this field.
hugginface_modelstr"EleutherAI/gpt-neo-125M"Not read anywhere in the current interface classes — no HuggingFaceInterface exists in this package. Kept for reference/future use.
gemini_modelstr"gemini-3-flash-preview"Default model_name for GeminiInterface.
mistral_modelstr"mistral-large-latest"Default model_name for MistralInterface.
groq_modelstr(field exists, default "llama-3.3-70b-versatile")Not actually referenced by GroqInterface — GroqInterface.__init__ hardcodes its own default model_name="llama-3.3-70b-versatile" (matching value, but read from the literal, not from config.groq_model).
ollama_modelstr"llama2"Default model_name for OllamaInterface.
ollama_base_urlstr"http://127.0.0.1:11434"Default base_url for OllamaInterface (overridable via the base_url kwarg).
time_outint200Read timeout (seconds) for Ollama's HTTP requests.
aws_modelstr(field exists, default matches)Not actually referenced by AWSInterface — it hardcodes its own default model_name="anthropic.claude-3-5-sonnet-20241022-v2:0" directly rather than reading config.aws_model.
aws_regionstr"us-east-1"Not actually referenced by AWSInterface — its region_name constructor parameter defaults to the literal "us-east-1" rather than config.aws_region.

[!WARNING] Some fields are defined but not wired up groq_model, aws_model, and aws_region exist on llm_config with values that happen to match each interface's own hardcoded defaults today, but the interfaces do not actually read from the shared llm_config instance for these particular fields — each subclass module hardcodes its own default. Changing llm_config.groq_model, for instance, will not change GroqInterface's default model. If you need a non-default model, pass model_name=... explicitly at construction time; don't rely on editing the dataclass for these three fields.

max_token / temperature and Python's default-argument evaluation

Every module that uses llm_config for method defaults does this pattern:

Python · 5 lines
config = llm_config()

class SomeInterface(BaseLLMInterface):
    def generate(self, prompt: str, max_tokens: int = config.max_token, temperature: float = config.temperature, **kwargs):
        ...

Because config = llm_config() runs once at import time, and Python evaluates default parameter values once at function-definition time, max_tokens's default is baked in as 2048 and temperature's as 0.3 the moment the module is imported. Reassigning fields on that module-level config object after import has no effect on already-compiled method defaults. To change behavior at runtime, pass max_tokens=/temperature= explicitly on each call, or construct your own llm_config(...) with different values before importing the provider modules (impractical in most cases) — in practice, just pass the values explicitly per call.

Gemini's extra, undocumented-by-dataclass settings

GeminiInterface hardcodes rate-limiting behavior that has no corresponding llm_config field:

  • _min_request_interval = 5.0 seconds between requests (to stay under Gemini's free-tier 15 req/min limit).
  • max_retries = 5 and base_delay = 5 (exponential backoff with jitter) for 503/UNAVAILABLE/429 errors.

These are not configurable via llm_config — they're literals inside gemini_interface.py. See GeminiInterface for details.

Ollama's extra constructor kwargs

OllamaInterface reads three settings from **kwargs rather than llm_config: base_url (defaults to config.ollama_base_url), auto_start (defaults to True), and server_start_wait (defaults to 10 seconds). See OllamaInterface.

Related pages