Architecture

Component diagram

Request lifecycle

HierarchicalRouter.route_query (see API reference) executes these steps in order:

Two-level routing

RoutingPipeline.run (see API reference) performs routing in two levels:

[!NOTE] Group scoring re-uses the route scorer RoutingPipeline._select_groups doesn't have a separate group-scoring algorithm — it wraps each RouteGroup in a synthetic Route proxy (using the group's intent_examples, intent_keywords, and pre-computed embeddings) and runs it through the same HybridScorer used for route-level scoring. This is explicit in the source: "Build a synthetic Route per group to reuse the scorer."

Scoring formula

HybridScorer.score_all (see API reference) computes, for each candidate route:

combined = (w_sem * semantic_score) + (w_kw * keyword_score) + (w_llm * llm_score)
effective_score = min(1.0, combined + route.score_bias)

where w_sem, w_kw, w_llm are ScoringConfig.semantic_weight / keyword_weight / llm_weight normalized to sum to 1 (dividing each by their total). If no LLMScorer is configured, w_llm is forced to 0 before normalization.

The LLM scorer is only invoked when the best raw semantic+keyword score across all candidates falls below ScoringConfig.fallback_confidence_threshold — this is a cost-control measure since LLM calls are the most expensive scoring signal.

Confidence tiers

ConfidenceEvaluator.evaluate (see API reference) maps the final combined score to a ConfidenceLevel:

should_route() returns True for HIGH, MEDIUM, and LOW — only NONE triggers the no-match path (fallback handler, or a failed HandlerResponse, or a raised LookupError if RouterConfig.raise_on_no_match is set).

Execution modes

Once a route (or ranked candidate list) is selected, ExecutionEngine.execute (see API reference) dispatches based on ExecutionConfig.mode:

Feedback loop

Caching

CacheManager wraps a single internal _LRUTTLCache keyed by a SHA-256 hash of the normalized (lower-cased, whitespace-collapsed) query string, storing (route_name, group_name) tuples. A cache hit lets route_query skip both EmbeddingProvider.encode and the entire RoutingPipeline — it goes straight to route.execute(request). See CacheManager.

Separately, EmbeddingProvider (in routing/pipeline.py) maintains its own in-memory dict cache of query→embedding, keyed by MD5 hash of the text — this is a distinct cache from CacheManager's route-decision cache and is not evicted by TTL/LRU (it only clears via EmbeddingProvider.clear_cache()).