Document Loaders

On this page

document_loaders is the document-ingestion module of the Zerda ecosystem. It converts files, web pages, directories, knowledge graphs, and tables into a single normalized unit — LoadedDocument — that the rest of the Fennec / RAG pipeline (chunking, embedding, indexing) can consume without caring what the original source format was.

This documentation is generated directly from static analysis of the document_loaders source package. Every signature, parameter, exception, and behavior described here is taken from the actual implementation. Where something cannot be determined from the code (e.g. version history, since no changelog or git metadata ships with this package), it is stated explicitly rather than guessed.

Why this module exists

RAG and LLM applications need text from many different sources — PDFs, Word docs, spreadsheets, scraped web pages, JSON exports, knowledge graphs. Each format has its own quirks (encodings, password protection, broken .doc files, rate-limited APIs, malformed CSV). document_loaders centralizes all of that handling behind one consistent contract:

Python · 2 lines
loader = SomeLoader(...)
docs: list[LoadedDocument] = loader.load()

Every loader, regardless of source type, returns a list of LoadedDocument objects with page_content (the text) and metadata (a dict describing where it came from).

Module map

ModulePurpose
base_loaderCore data model (LoadedDocument) and abstract base classes
config_loaderAll loader configuration dataclasses + LoaderType enum
text_loaderPlain text (.txt) and Markdown (.md) files
pdf_loaderPDF files via a fitz → pdfplumber → pypdf fallback chain
docx_loaderMicrosoft Word .docx (and legacy .doc via LibreOffice/textract)
csv_loaderCSV/TSV and Excel spreadsheets
json_loaderJSON and JSON Lines (.jsonl/.ndjson)
html_loaderHTML files and raw HTML strings
web_loaderSingle and concurrent multi-URL web fetching
directory_loaderRecursive directory ingestion with concurrency
auto_loaderFormat auto-detection — the recommended entry point
multimodal_loaderOCR for images, mixed-content PDFs, HTML table extraction
graph_loaderKnowledge graphs (JSON node-link, GraphML, GEXF, edge CSV)
table_loaderFormat-agnostic table extraction (Excel/CSV/PDF/HTML/Markdown)

Where to go next

Bilingual codebase

The source code mixes English docstrings with Arabic comments and docstrings (notably in graph_loader.py and table_loader.py), reflecting the project's focus on first-class Arabic/RTL document support. This documentation is written in English throughout, but Arabic text in code examples (e.g. sample node labels, CSV rows) is preserved as it appears in the source.