ZerdaZerda

Router

On this page

A hierarchical, hybrid-scoring query router for LLM applications.

router (part of the Zerda ecosystem) decides which piece of code should handle a user's query — a RAG pipeline, a tool call, a chat handler, or anything else you register — using a two-level decision process:

  1. Group selection — pick the best-matching RouteGroup (e.g. rag, tools, chat) based on the group's intent examples and keywords.
  2. Route selection — within that group, pick the best-matching Route using a hybrid of semantic similarity, keyword signals, and (optionally) an LLM classifier.

Every decision is scored, cached, logged, and fed back into an adaptive bias system so routing accuracy improves as your application is used.

Why a router?

LLM applications typically need to dispatch a user's message to one of many possible handlers — retrieve from a knowledge base, call a tool, run code, or simply chat. Doing this well means answering three questions on every request:

  • Which handler is the best fit? — combining embedding similarity with cheap keyword heuristics avoids paying for an LLM call on every message.
  • How confident are we? — a low-confidence decision should behave differently from a high-confidence one (route with a warning, fall back to an LLM classifier, or give up entirely).
  • What happens after? — retries, timeouts, streaming, and multi-route execution all need a consistent contract regardless of which handler was chosen.

router answers all three with a single, typed API surface.

Core capabilities

CapabilityDescription
Hierarchical routingTwo-level group → route selection reduces the search space and lets you organize routes by intent domain.
Hybrid scoringCombines semantic (embedding) similarity, keyword matching, and an optional LLM fallback classifier into one weighted score.
Confidence tiersEvery decision is classified as HIGH / MEDIUM / LOW / NONE confidence, driving routing, fallback, or clarification behavior.
Execution modesSINGLE, SEQUENTIAL (try top-k, stop on success), and PARALLEL (run top-k concurrently, aggregate).
Sync + asyncEvery routing and execution path has a matching async variant.
StreamingToken-level streaming from the winning handler, plus an event-based streaming API for driving UIs.
Adaptive feedbackAn exponential-moving-average feedback loop nudges route scores based on observed success/failure.
CachingA thread-safe LRU+TTL cache skips embedding and scoring entirely for repeated queries.
ObservabilityStructured JSON logging and rolling-window metrics (latency, match rate, confidence distribution, per-route stats) out of the box.
Tool chainingExecute a fixed sequence of routes as a pipeline, passing each result into the next route's context.

A minimal example

Python · 20 lines
from zerda.router import HierarchicalRouter, RouteGroup, Route
from zerda.router import BaseHandler, HandlerRequest, HandlerResponse

class GreetHandler(BaseHandler):
    def handle(self, request: HandlerRequest) -> HandlerResponse:
        return HandlerResponse.ok(f"Hello! You said: {request.query}")

router = HierarchicalRouter()

chat = RouteGroup("chat", "General conversation")
chat.add_route(Route(
    name="greet",
    description="Greet the user",
    handler=GreetHandler(),
    examples=["hello", "hi there", "good morning"],
))
router.register_group(chat)

result = router.route_query("Hey, how are you?")
print(result.content)

Continue to Installation to get set up, or jump straight to the Quickstart for a fuller walkthrough.

Documentation map

[!NOTE] Scope of this documentation This documentation covers the router package as found in the analyzed source tree (router/). It does not cover other zerda packages (chain, embeddings, document_loaders, etc.) except where router directly imports from them.