ZerdaZerda

Architecture

On this page

Table of Contents

Overview

The library is organized into three layers:

LayerComponentsResponsibility
ModelsDocumentChunk, Document, ChunkMetadata, enumsImmutable data contracts
ChunkersSemanticChunker, AdaptiveChunker, StructureAwareChunker, ContextAwareChunker, ArabicTextChunker, MultilanguageTextChunkerText → List[DocumentChunk]
OrchestrationChunkManager, QueryPatternAnalyzer, MetadataEnricher, DeduplicatorPipeline management

Supporting components (EmbeddingProvider, DocumentStructureParser, ArabicNormalizer) are used internally by chunkers.

Component Hierarchy

Data Flow

Chunker Hierarchy

Module Dependency Graph

Extension Points

Custom Chunker

Subclass BaseChunker and implement _chunk_impl:

Python · 22 lines
from zerda.chunks import BaseChunker
from zerda.chunks import DocumentChunk, ChunkMetadata, ChunkType
import uuid

class MyChunker(BaseChunker):
    def _chunk_impl(self, text: str, doc_id: str, source: str):
        # Your splitting logic here
        parts = text.split("\n---\n")
        chunks = []
        for i, part in enumerate(parts):
            if part.strip():
                chunks.append(DocumentChunk(
                    text=part.strip(),
                    chunk_id=str(uuid.uuid4()),
                    doc_id=doc_id,
                    metadata=ChunkMetadata(
                        source=source,
                        position=i,
                        chunk_type=ChunkType.SECTION,
                    )
                ))
        return chunks

The public chunk() method inherited from BaseChunker automatically calls _finalize() to set position and total_chunks.

Custom Text Splitter

Subclass TextSplitter and implement split_text:

Python · 5 lines
from zerda.chunks import TextSplitter

class MyTextSplitter(TextSplitter):
    def split_text(self, text: str):
        return [t.strip() for t in text.split("|||") if t.strip()]

Related pages