Integrate Diverse File Types for LLM Data Ingestion
The default file-format reader package behind LlamaIndex's SimpleDirectoryReader, covering 22 file types.
Why it matters
Seamlessly ingest and process a wide variety of file formats into your LLM applications. This asset provides robust readers for documents, presentations, images, and more, enabling comprehensive data indexing and retrieval.
Outcomes
What it gets done
Load data from PDF, DOCX, PPTX, and EPUB files.
Extract information from images, including captions and tabular data.
Process structured data from CSV and XML files.
Ingest content from HTML, Markdown, and MBOX formats.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/li-reader-readers-file | bash Overview
LlamaIndex Readers Integration: File
The default LlamaIndex reader package behind SimpleDirectoryReader, providing 22 format-specific loaders for PDF, Word, images, CSV, PowerPoint, HTML, Markdown, and more. Use it whenever loading a mixed-format directory into LlamaIndex, registering each reader against its file extension in file_extractor rather than calling it standalone.
What it does
llama-index-readers-file is the default integration providing the loaders used within SimpleDirectoryReader. It ships 22 format-specific readers: DocxReader, HWPReader, PDFReader, EpubReader, FlatReader, HTMLTagReader, ImageCaptionReader, ImageReader, ImageVisionLLMReader, IPYNBReader, MarkdownReader, MboxReader, PptxReader, PandasCSVReader, VideoAudioReader, UnstructuredReader, PyMuPDFReader, ImageTabularChartReader, XMLReader, PagedCSVReader, CSVReader, and RTFReader - covering Word, Hangul, PDF, ebook, plain text, HTML, images, Jupyter notebooks, Markdown, mbox mail archives, PowerPoint, CSV/pandas, audio/video, unstructured documents, XML, and RTF.
When to use - and when NOT to
Use this package whenever you're loading a directory of mixed file types into LlamaIndex and want a ready-made parser per extension instead of writing your own for each format. Register the reader you need against its file extension in SimpleDirectoryReader's file_extractor mapping rather than calling it standalone, since that's the pattern the whole package is built around.
Inputs and outputs
Each reader is instantiated on its own, then wired into file_extractor as a dict mapping an extension to the parser instance, and SimpleDirectoryReader.load_data() does the rest:
parser = PDFReader()
file_extractor = {".pdf": parser}
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
The same pattern applies per extension - .docx to DocxReader, .hwp to HWPReader, .epub to EpubReader, .txt to FlatReader, .html to HTMLTagReader, .jpg/.jpeg/.png to ImageReader, .ipynb to IPYNBReader, .md to MarkdownReader, .mbox to MboxReader, .csv to PandasCSVReader or CSVReader, and so on for the rest of the covered formats. Beyond ImageReader for plain image loading, the package also separates out ImageCaptionReader (generates a caption for an image), ImageVisionLLMReader (uses a vision-capable LLM to describe an image), and ImageTabularChartReader (extracts data from images of tables or charts) as distinct readers, rather than folding that logic into one image loader. VideoAudioReader covers audio and video files, PyMuPDFReader is an alternative PDF backend to PDFReader, and PagedCSVReader offers page-based CSV loading as an alternative to PandasCSVReader and the plain CSVReader.
Integrations
PptxReader has notably deeper configuration than the others: its basic usage already extracts text, tables, charts, and speaker notes, and its advanced options add extract_images (enables image captioning), context_consolidation_with_llm (uses an LLM to synthesize slide content), num_workers (parallel processing), batch_size (how many slides each worker processes per batch), and raise_on_error (whether a parsing failure raises instead of failing silently). Install the whole package with pip install llama-index-readers-file.
Who it's for
Developers building LlamaIndex ingestion pipelines over mixed-format document sets who want per-format parsing without hand-writing 22 separate loaders. Once the package is installed, any of its readers can simply be imported and used directly, so adopting one additional format is usually a one-line import plus one new file_extractor entry, not a new dependency to research and integrate.
Source README
LlamaIndex Readers Integration: File
pip install llama-index-readers-file
This is the default integration for different loaders that are used within SimpleDirectoryReader.
Provides support for the following loaders:
- DocxReader
- HWPReader
- PDFReader
- EpubReader
- FlatReader
- HTMLTagReader
- ImageCaptionReader
- ImageReader
- ImageVisionLLMReader
- IPYNBReader
- MarkdownReader
- MboxReader
- PptxReader
- PandasCSVReader
- VideoAudioReader
- UnstructuredReader
- PyMuPDFReader
- ImageTabularChartReader
- XMLReader
- PagedCSVReader
- CSVReader
- RTFReader
Installation
pip install llama-index-readers-file
Usage
Once installed, You can import any of the loader. Here's an example usage of one of the loader.
from llama_index.core import SimpleDirectoryReader
from llama_index.readers.file import (
DocxReader,
HWPReader,
PDFReader,
EpubReader,
FlatReader,
HTMLTagReader,
ImageCaptionReader,
ImageReader,
ImageVisionLLMReader,
IPYNBReader,
MarkdownReader,
MboxReader,
PptxReader,
PandasCSVReader,
VideoAudioReader,
UnstructuredReader,
PyMuPDFReader,
ImageTabularChartReader,
XMLReader,
PagedCSVReader,
CSVReader,
RTFReader,
)
#### PDF Reader with `SimpleDirectoryReader`
parser = PDFReader()
file_extractor = {".pdf": parser}
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
#### Docx Reader example
parser = DocxReader()
file_extractor = {".docx": parser}
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
#### HWP Reader example
parser = HWPReader()
file_extractor = {".hwp": parser}
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
#### Epub Reader example
parser = EpubReader()
file_extractor = {".epub": parser}
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
#### Flat Reader example
parser = FlatReader()
file_extractor = {".txt": parser}
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
#### HTML Tag Reader example
parser = HTMLTagReader()
file_extractor = {".html": parser}
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
#### Image Reader example
parser = ImageReader()
file_extractor = {
".jpg": parser,
".jpeg": parser,
".png": parser,
} # Add other image formats as needed
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
#### IPYNB Reader example
parser = IPYNBReader()
file_extractor = {".ipynb": parser}
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
#### Markdown Reader example
parser = MarkdownReader()
file_extractor = {".md": parser}
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
#### Mbox Reader example
parser = MboxReader()
file_extractor = {".mbox": parser}
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
#### Pptx Reader example
#### Basic usage - extracts text, tables, charts, and speaker notes
parser = PptxReader()
#### Advanced usage - control parsing behavior
parser = PptxReader(
extract_images=True, # Enable image captioning
context_consolidation_with_llm=True, # Use LLM for content synthesis
num_workers=4, # Parallel processing
batch_size=10, # Slides processed per worker batch
raise_on_error=True, # Raise value error if file_parsing is not successful
)
file_extractor = {".pptx": parser}
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
#### Pandas CSV Reader example
parser = PandasCSVReader()
file_extractor = {".csv": parser} # Add other CSV formats as needed
documents = SimpleDirectoryReader(
"./data", file_extractor=file_extractor
).load_data()
#### PyMuPDF Reader example
parser = PyMuPDFReader()
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.