Quickstart

This walks through the minimal end-to-end flow: define some chunks, index them, and run a query — using only what is actually implemented in ContextEngine.

1. Define your chunks

Any object works as a "chunk" as long as it exposes a text attribute (or page_content, for LangChain-style documents) and ideally a doc_id/id and metadata. This duck-typing is implemented in ScoredChunk's property accessors — see models.ScoredChunk.

from dataclasses import dataclass, field
from typing import Dict, Any

@dataclass
class Chunk:
    doc_id: str
    text: str
    metadata: Dict[str, Any] = field(default_factory=dict)

chunks = [
    Chunk(doc_id="ml-101", text="التعلم الآلي هو فرع من الذكاء الاصطناعي يهتم ببناء أنظمة تتعلم من البيانات."),
    Chunk(doc_id="ml-102", text="Machine learning is a subfield of AI focused on building systems that learn from data."),
    Chunk(doc_id="ml-103", text="من أشهر خوارزميات التعلم الآلي: الانحدار الخطي، الشجرات القرارية، والشبكات العصبية."),
]

2. Provide an embedding function (optional but recommended)

ContextEngine.index() accepts any callable that maps str -> np.ndarray. If you don't have one, vector retrieval will simply produce no results and the engine falls back to BM25 keyword retrieval — it will not raise an error.

import numpy as np

def fake_embed(text: str) -> np.ndarray:
    # Replace with a real embedding model (OpenAI, sentence-transformers, etc.)
    rng = np.random.default_rng(abs(hash(text)) % (2**32))
    return rng.random(64)

3. Build and index the engine

from zerda.context import ContextEngine, ContextEngineConfig

engine = ContextEngine(config=ContextEngineConfig())
engine.index(chunks=chunks, embed_fn=fake_embed)

index() builds a VectorRetriever, KeywordRetriever, and MetadataRetriever internally and wires them into a HybridRetriever — see engine.ContextEngine.index.

[!TIP] Skip the hand-rolled Chunk class and embedding function If you already use (or are willing to use) zerda.chunks for chunking and zerda.vector_database for storage, you can skip steps 1–3 above entirely and let the engine do both natively — no custom Chunk dataclass, no fake_embed:

from zerda.chunks import StructureAwareChunker, ChunkConfig
from zerda.vector_database import ChromaVectorDatabase
from zerda.context import ContextEngine
from zerda.embeddings import HugginFaceEmbedder

my_embedder = HugginFaceEmbedder()
chunks = StructureAwareChunker(ChunksConfig)
engine = ContextEngine()
engine.index_documents(
    texts     = ["التعلم الآلي هو فرع من الذكاء الاصطناعي...", "..."],
    chunker   = chunks,
    vector_db = ChromaVectorDatabase(embedder=my_embedder),  # any real embedder with .encode()
)

See engine.ContextEngine.index_documents and the native integration recipe for the full walkthrough, including how metadata filters are resolved per-backend.

4. Run a query

result = engine.run("ما هو التعلم الآلي؟")

print(result.context)
print(result.get_stats())

Typical output shape (exact text depends on your chunks and embed function):

📚 المعلومات المسترجعة:

[المصدر: ml-101]
التعلم الآلي هو فرع من الذكاء الاصطناعي يهتم ببناء أنظمة تتعلم من البيانات.
---
[المصدر: ml-103]
من أشهر خوارزميات التعلم الآلي: الانحدار الخطي، الشجرات القرارية، والشبكات العصبية.

─── نهاية السياق ───

{
    'pipeline_ms': 1.42,
    'total_retrieved': 2,
    'total_after_filter': 2,
    'total_after_rank': 2,
    'total_included': 2,
    'context_length': 238,
    'guard_passed': True,
    'guard_warnings': [],
    'query_intent': 'factual',
    'query_complexity': 'low',
    'context_budget': 800,
}

5. Feed it to your LLM

result.context is a plain string — pass it straight into your prompt template:

prompt = f"""Use the following context to answer the question.

{result.context}

Question: ما هو التعلم الآلي؟
Answer:"""

What just happened?

Every one of these steps maps directly onto a pipeline stage. Read Architecture next to see exactly what run() does internally, in order.