ZerdaZerda

core Module Documentation

On this page

core is the retrieval-augmented-generation (RAG) engine at the heart of the Zerda library. It provides a bring-your-own-components architecture (vector database, LLM, chunker, context builder) wrapped in a batteries-included pipeline: query expansion, retrieval caching, LLM/heuristic re-ranking, bilingual (Arabic/English) prompt routing, document versioning, and RAGAS-style evaluation.

What's in this package

ModuleResponsibility
rag_systemThe concrete, production-ready RAGSystem implementation — ingestion, retrieval, generation, sync + async APIs, persistence.
base_rag_systemBaseRAGSystem — the abstract contract every RAG implementation must follow.
rag_configRAGConfig and RerankerConfig dataclasses, plus the RerankMode enum — central configuration for the whole pipeline.
rerankerReranker — heuristic / LLM / hybrid re-ranking with near-duplicate filtering.
prompt_routerPromptRouter — detects language and question type, builds tailored prompts.
query_expansionQueryExpander — LLM- and synonym-table-based query expansion, plus merge_retrieval_results.
retrieval_cacheRetrievalCache — thread-safe TTL/LRU cache for retrieval results.
evaluationRAGEvaluator and related dataclasses — RAGAS-style answer/context quality scoring.
document_versioningDocumentVersionManager — diff-based re-indexing and version history/rollback.
exceptionsFennecError and the full exception hierarchy.
loggersetup_logging, get_logger, FennecLogger — centralized logging.

Where to start

  • Want the big picture first? Read Architecture.
  • Tuning retrieval/generation behavior? See Configuration.
  • Looking for a specific class or function? Go to the API Overview.
  • Practical, task-oriented recipes live in the Cookbook.

Design philosophy (as observed in the source)

  • Bring your own components. RAGSystem does not implement a vector database, an LLM client, or a chunker — it orchestrates whatever objects you pass in, as long as they satisfy a small duck-typed contract (see RAGSystem).
  • Fail soft on generation, fail loud on retrieval. RAGSystem.generate() is designed to never raise — errors are turned into readable strings — while RAGSystem.retrieve() raises RAGRetrievalError on unexpected failures.
  • Bilingual by default. Arabic and English are treated as first-class citizens throughout — language detection, prompt templates, few-shot examples, and the synonym table all ship with both languages.
  • Sync and async in parallel. Every major entry point (add_documents, retrieve, generate) has an a-prefixed async counterpart (aadd_documents, aretrieve, agenerate), plus astream for token streaming.
  • Everything is optional. Reranking, query expansion, prompt routing, and retrieval caching can each be independently enabled or disabled via RAGConfig.

Related pages