ZerdaZerda

Cookbook: Caching & Invalidation

On this page

Practical patterns for working with RetrievalCache in a production RAGSystem.

Recipe: Tune cache size and TTL for your traffic pattern

Python · 7 lines
from zerda.rag.core import RAGConfig

config = RAGConfig(
    enable_retrieval_cache=True,
    cache_maxsize=2000,   # higher for high-cardinality query traffic
    cache_ttl=900,        # 15 minutes — balance freshness vs. hit rate
)

Set cache_ttl=0 (or any value <= 0) to disable time-based expiry entirely — RAGSystem translates this to ttl=None internally.

Recipe: Monitor hit rate in production

Python · 4 lines
stats = rag.get_cache_stats()
if stats and stats["hit_rate"] < 0.2:
    logger.warning("Low cache hit rate (%.1f%%) — consider raising cache_ttl "
                    "or investigating query diversity.", stats["hit_rate"] * 100)

Recipe: Invalidate on external content updates

If a document is modified by a process other than RAGSystem.add_documents() / .remove_document() — for example, a direct write to the vector DB — the retrieval cache won't know about it automatically:

Python
rag.invalidate_cache()   # clears everything; simplest, safest option

add_documents() and — indirectly, via remove_document()'s stats update — already keep the cache roughly consistent for changes made through RAGSystem's own API; invalidate_cache() exists for everything else.

Recipe: Avoid cache-clearing storms during bulk ingestion

Every successful call to add_documents() clears the entire retrieval cache (see RAGSystem.add_documents side effects). If you're ingesting documents one at a time in a loop, you're also fully invalidating the cache on every iteration:

Python · 6 lines
# Bad: clears the cache N times
for doc in documents:
    rag.add_documents([doc])

# Better: batch, so the cache is only cleared once
rag.add_documents(documents)

Recipe: Skip caching for one-off diagnostic queries

The retrieval cache has no per-call opt-out — it's controlled only via RAGConfig.enable_retrieval_cache at construction time. For a one-off query you don't want cached (e.g. an internal health check), call vector_db.search() directly instead of going through rag.retrieve():

Python
raw_results = rag.vector_db.search("health check probe", top_k=1, score_threshold=0.0)

Recipe: Cold-start warming

Since retrieve()'s cache lookup happens before query expansion and reranking, "warming" the cache for known-frequent queries just means calling retrieve() for them once at startup:

Python · 3 lines
COMMON_QUERIES = ["how do I reset my password", "what is the refund policy"]
for q in COMMON_QUERIES:
    rag.retrieve(q)

See also: RetrievalCache API reference for the full method list and internal eviction/expiry behavior.

Related pages