ZerdaZerda

Migration Guide

On this page

From other document-loading libraries

If you're coming from a similar library (e.g. LangChain's document loaders, LlamaIndex readers), the concepts map fairly directly:

Concept elsewhereHere
DocumentLoadedDocument — same page_content/metadata shape, plus a deterministic doc_id
Format-specific loader classesSame pattern — one class per format, all implementing load()
Directory/glob loaderDirectoryLoader
Auto-detecting loaderAutoLoader
Text splitter integrationload_and_split(text_splitter) — duck-typed on split_text(str) -> List[str], bring your own splitter

Key differences to watch for

  • No plugin/registry system. There's no register_loader() mechanism — extending format support means subclassing DirectoryLoader/AutoLoader yourself (see Advanced Usage).
  • Deterministic doc_id. Unlike libraries that don't assign IDs or use random/UUID-based IDs, doc_id here is a content hash — identical content from the same source/page always gets the same ID across runs. Useful for dedup, but also means editing a document's metadata alone (without changing content, source, or page_number) does not change its ID.
  • GraphLoader/TableLoader are not part of auto-detection. If migrating from a system where graph/table loaders were unified with general format detection, note these two require explicit instantiation here.
  • No async API. If migrating from an async-first library, everything here is synchronous; concurrency is thread-pool-based (DirectoryLoader, MultiURLLoader) rather than asyncio-based.

Migrating between versions of this package

No changelog, version tags, or git history were available in the analyzed source tree, so specific version-to-version migration notes cannot be determined. The source code does contain several in-line comments describing bugs that were fixed in the current version relative to some earlier, unversioned state of the code:

  • LoadedDocument.__post_init__ — doc_id generation changed from time-based (non-deterministic) to content-hash-based (deterministic). If you have data indexed under the old time-based IDs, re-indexing with the current version will assign different IDs to the same content, breaking any ID-based deduplication or upsert logic that assumed stability. Plan a one-time re-index if migrating from a version with the old ID scheme.
  • CSVLoader.lazy_load() — encoding-retry logic changed from a recursive retry (risk of silent loops/duplicates) to a fixed two-step [configured, "latin-1"] sequence. No output-shape change for successful loads; only affects edge-case failure behavior.
  • DirectoryLoader._collect_files() — recursive/non-recursive glob pattern handling was fixed (previously both branches incorrectly stripped the "**/" prefix). If you were relying on the previous (buggy) behavior for a specific glob pattern, re-verify your glob_pattern/recursive settings produce the file set you expect under the current implementation.
  • DocxLoader._load_legacy_doc() — .doc handling changed from an incorrect PDF-reader-based attempt to a proper LibreOffice/textract conversion chain. If you previously worked around this by pre-converting .doc files yourself, that workaround is no longer necessary (though still harmless).

If your project's own changelog or commit history tracks when these fixes landed, cross-reference that against this list to understand what changed for your specific upgrade path — this documentation cannot supply exact version numbers since none were present in the analyzed source.

Related pages