FAQ
On this page
Which loader should I use for a mixed corpus of file types?
Use AutoLoader for a single file/URL/directory of unknown-in-advance type, or DirectoryLoader directly when you're scanning a whole folder. Both cover text, Markdown, PDF, Word, CSV, Excel, JSON, JSONL, and HTML automatically.
Does this package support scanned/image-only PDFs?
Yes, via PDFMultimodalLoader with ocr_images=True (the default), which runs Tesseract OCR on embedded images per page in addition to extracting any digital text. Plain PDFLoader does not do OCR — pages with no digital text are simply skipped.
Why does PDFLoaderConfig.extract_images not seem to do anything?
It's declared in the config dataclass but never read anywhere in pdf_loader.py — it has no effect in the current implementation. Use PDFMultimodalLoader for image-related PDF processing.
How do I load a password-protected PDF or a .doc file that needs a converter?
PDFLoader("file.pdf", password="secret").load()
For legacy .doc files, DocxLoader automatically tries LibreOffice conversion, then textract, raising a clear ImportError with remediation steps if neither is available. See DocxLoader.
Why did my CSV row disappear from the output?
CSVLoader/ExcelLoader/TableLoader all silently skip rows whose resulting content is empty after column filtering and stripping — this isn't a bug, but it means row counts in output may not match the source file's row count. Check whether your content_columns filter is excluding the only non-empty columns in some rows.
My Arabic/non-UTF-8 text looks garbled after loading. Why?
Most text-based loaders (TextLoader, MarkdownLoader, CSVLoader) fall back to latin-1 decoding if the configured encoding fails and chardet either isn't installed or fails to detect a better match. latin-1 never raises an error, but it can silently produce mojibake for non-Latin-1 text. Install chardet (pip install chardet) and check metadata["encoding_used"] on the resulting documents to diagnose.
How do I control which columns become content vs. metadata in a CSV/Excel file?
Use content_columns and metadata_columns — see CSVLoaderConfig and the CSV/Excel API page.
What's the difference between CSVLoader, ExcelLoader, and TableLoader?
CSVLoader/ExcelLoader always produce one document per row. TableLoader is format-agnostic (Excel, CSV, PDF, HTML, Markdown) and lets you choose whole-table vs. per-row granularity, plus a choice of four output formats (Markdown/CSV/JSON/text) for whole-table documents.
Are GraphLoader and TableLoader auto-detected by AutoLoader?
No. Neither is wired into EXTENSION_MAP/AutoLoader/DirectoryLoader's dispatch tables — always import and instantiate them explicitly.
Can I pass a GraphLoaderConfig/TableLoaderConfig with custom include_nodes/output_format settings?
Yes, but be careful: both GraphLoader and TableLoader have keyword-only constructor arguments (include_nodes/include_edges for GraphLoader; output_format/per_row for TableLoader) that always overwrite the equivalent fields on a passed-in config= object. Set those specific fields as direct constructor kwargs, not just inside the config object. See the warnings on the Graph and Table API pages.
Does DirectoryLoader.lazy_load() truly stream in concurrent mode?
No — in concurrent mode (use_multithreading=True, the default), it waits for all files to finish loading before yielding anything, despite being a generator method. For true progressive yielding, use sequential mode (use_multithreading=False). See DirectoryLoader.
How do I chunk/split loaded documents?
Every loader's load_and_split(text_splitter) method accepts any object with a split_text(str) -> List[str] method. There's no bundled text splitter in this package — bring your own (from the wider Fennec ecosystem, LangChain, or your own implementation). See Integrations.
Is there caching or deduplication built in?
There's no caching of load results — every load() call re-reads and re-parses from scratch. However, LoadedDocument.doc_id is deterministic (content-hash-derived), so you can build deduplication/incremental-indexing on top by tracking which doc_ids you've already processed. See LoadedDocument.
Does the package use asyncio?
No. All concurrency (DirectoryLoader, MultiURLLoader) uses concurrent.futures.ThreadPoolExecutor, not asyncio. There is no async/await API anywhere in this codebase.
What happens if I load a file with an unsupported extension via AutoLoader?
AutoLoader.load() raises ValueError with a message listing all extensions currently in EXTENSION_MAP.
Where can I find the full list of every configuration option?
Configuration Reference — every dataclass field, with type, default, and purpose.