ZerdaZerda

Tutorial: Production Deployment Checklist

On this page

A checklist-style walkthrough for taking a router-based application to production, based on the configuration surface and behaviors documented across this site.

1. Choose an execution mode deliberately

  • ExecutionMode.SINGLE (default) — lowest latency, no fallback if the top route's handler fails.
  • ExecutionMode.SEQUENTIAL — tries the next-best candidate on failure; good default for production if you have overlapping/fallback-capable routes.
  • ExecutionMode.PARALLEL — highest cost (executes multiple candidates concurrently), useful when you want to race multiple strategies and merge/pick the best.

See Configuration → ExecutionConfig and note the sync/async behavioral differences documented in ExecutionEngine (async paths do not retry; async parallel discards all results on timeout rather than keeping partial results).

2. Set explicit confidence thresholds for your domain

The defaults (high=0.85, medium=0.60, fallback=0.40) are reasonable starting points but should be tuned against your actual route examples and query distribution. Log result.trace.candidates scores during a staging period and adjust thresholds based on observed separation between correct and incorrect top matches.

3. Always set a fallback handler

Python
router.set_fallback(ClarifyOrEscalateHandler())

Without one, _handle_no_match either raises LookupError (if raise_on_no_match=True) or returns a failed RoutingResult — neither is a good default end-user experience. See HierarchicalRouter._handle_no_match.

4. Wire up observability

Python · 10 lines
from zerda.router import ObservabilityConfig, LogLevel

config = RouterConfig(
    observability=ObservabilityConfig(
        log_level=LogLevel.INFO,
        structured_logging=True,   # JSON logs — pipe into your log aggregator
        slow_route_ms=800.0,       # tune to your latency SLO
        metrics_window=2000,
    ),
)

Expose router.metrics() and router.cache_stats() on a health/metrics endpoint (e.g. a /internal/router-metrics route in your web framework) so you can monitor match rate, latency percentiles, and cache hit rate over time — see MetricsCollector.snapshot.

5. Decide on caching TTL deliberately

CacheConfig.ttl_seconds (default 300) controls how long a route decision is trusted. Since most route/group mutation methods already clear the cache, TTL mainly guards against stale decisions after score_bias drifts significantly via feedback. If your feedback loop adapts quickly (low min_samples_to_adapt, high alpha), consider a shorter TTL.

6. Persist feedback state across deploys

Python
config.feedback.persist_path = "/var/lib/myapp/router_feedback.json"

Use HierarchicalRouter as a context manager (or call router._feedback.save() on shutdown) so learned biases survive restarts/redeploys rather than resetting to neutral every time.

7. Guard the embedding model load

HierarchicalRouter() construction loads the sentence-transformer model synchronously and can raise ImportError if sentence-transformers isn't installed. In containerized deployments, bake the model into the image (or pre-warm a shared cache volume) to avoid a slow, network-dependent cold start on every container boot.

8. Decide your LLM fallback posture

If using RouterConfig.llm_model, remember LLMScorer fails silently (returns empty scores) on any error, including missing openai, bad credentials, or rate limiting — there's no built-in alerting for LLM fallback failures. Add your own monitoring around this if the LLM tier is load-bearing for your accuracy targets.

9. Test the no-match and disabled-route paths explicitly

Python · 7 lines
# Confirm your fallback handler behaves correctly:
result = router.route_query("completely unrelated gibberish query xyz123")
assert not result.matched

# Confirm disabling a route doesn't break routing:
router.get_route("some_route").disable()
result = router.route_query("a query that would normally hit some_route")

Related pages