ZerdaZerda

GroqInterface

On this page

llm.GroqInterface — sync & async Groq interface with native token streaming.

Groq exposes an OpenAI-compatible API, and its Python SDK (groq) mirrors the openai SDK's client surface almost exactly — this class is structurally identical to OpenAIInterface, with groq.Groq / groq.AsyncGroq in place of openai.OpenAI / openai.AsyncOpenAI.

Constructor

Python · 5 lines
GroqInterface(
    model_name: str = "llama-3.3-70b-versatile",
    api_key: Optional[str] = None,
    **kwargs,
)

Creates both groq.Groq (sync) and groq.AsyncGroq (async) clients eagerly. Raises ImportError("install groq: pip install groq") if the groq package is missing.

[!NOTE] Default model is hardcoded, not read from llm_config llm_config.groq_model exists (also "llama-3.3-70b-versatile") but GroqInterface.__init__'s default value is a separate literal, not a reference to config.groq_model. The values happen to match today; see Configuration for the full list of fields like this.

ParameterTypeDefaultNotes
model_namestr"llama-3.3-70b-versatile"Per the source docstring, other examples include llama-3.1-8b-instant, mixtral-8x7b-32768, gemma2-9b-it. whisper-large-v3 is listed as a supported Groq model but is an audio model — not applicable to this text-generation interface.
api_keyOptional[str]NonePassed to both clients; SDK's own env-var fallback applies if None.
**kwargs——Stored via base class.

generate(messages=None, prompt=None, max_tokens=2048, temperature=0.3, tools=None, tool_choice=None, **kwargs) -> Union[str, dict]

Identical contract to OpenAIInterface.generate: messages takes priority over prompt; a missing prompt/messages triggers _build_messages's ValueError, but it's caught inside this same method (not propagated) and folded into the return value; returns r.choices[0].message.content on success or {"error": str(e)} on failure; other exceptions are logged via logger.error and swallowed the same way. tools, when given, is converted via llm.tools.to_groq_tools — a plain alias of to_openai_tools, since Groq's function-calling dialect mirrors OpenAI's exactly — and a resulting tool call is parsed with the same parse_openai_tool_calls helper OpenAI uses, returning the same {"content", "tool_calls", "stop_reason"} dict. See Tool Calling.

Example

Python · 4 lines
from zerda.llm import GroqInterface

llm = GroqInterface(api_key="gsk_...")
print(llm.generate(prompt="مرحباً!"))

generate_async(...)

Same contract, using self._async_client.chat.completions.create(...) with await. Same tools/tool_choice handling as generate.

astream(...) -> AsyncIterator[str]

Native streaming via stream=True, yielding chunk.choices[0].delta.content as each arrives — same pattern as OpenAI's. Errors yield f"Error: {e}" as stream content instead of raising. Does not reconstruct tool-call deltas — see Tool Calling § Streaming limitation.

cleanup() / acleanup()

Same pattern as OpenAIInterface: closes sync/async client, logging (not raising) on failure.

Related pages