Extract Document Data for RAG
Docling Reader is a LlamaIndex data reader that extracts PDF, DOCX, HTML, and other document types into Markdown or JSON format.
Why it matters
Leverage Docling to efficiently extract text and structure from various document types (PDF, DOCX, HTML) for seamless integration into LlamaIndex pipelines, powering RAG and QA applications.
Outcomes
What it gets done
Extract data from PDF, DOCX, and HTML files.
Export extracted data as Markdown or JSON.
Integrate document data into LlamaIndex for RAG/QA.
Combine with SimpleDirectoryReader for batch processing.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/li-reader-readers-docling | bash Overview
Docling Reader
Docling Reader is a LlamaIndex data reader that extracts content from PDF, DOCX, HTML, and other document types into Markdown or JSON format. It uses the Docling engine to enable fast document parsing for LlamaIndex pipelines. Use Docling Reader when working with LlamaIndex pipelines that need to ingest mixed document formats. It's ideal for converting documents to Markdown for downstream processing or preserving native structure in JSON when paired with a Docling Node Parser.
What it does
Docling Reader is a LlamaIndex data reader that uses Docling to extract content from PDF, DOCX, HTML, and other document types into Markdown or JSON-serialized Docling format. It enables fast document extraction for use in LlamaIndex pipelines.
When to use - and when NOT to
Use Docling Reader when you need to ingest diverse document formats (PDF, DOCX, HTML) into LlamaIndex pipelines. It's ideal when you want Markdown output for downstream processing or need to preserve Docling's native document structure in JSON format.
Do not use Docling Reader if you only need plain text extraction without structured parsing, as simpler text extractors may suffice. Avoid JSON export mode unless you plan to use a Docling Node Parser in your pipeline, as the native format requires appropriate parsing to be useful.
Inputs and outputs
You provide file paths (local or remote URLs) to documents in supported formats like PDF, DOCX, or HTML. The reader can process single files or work with SimpleDirectoryReader for batch processing of document directories.
You receive documents containing either Markdown-formatted text (default) or JSON-serialized Docling format. Metadata includes file path, name, type, size, creation date, and last modified date.
Integrations
Docling Reader integrates with Docling (the underlying extraction engine from DS4SD) for document parsing. It works natively within LlamaIndex pipelines and can be combined with SimpleDirectoryReader for directory-based document ingestion. When using JSON export, it requires a Docling Node Parser to appropriately parse the native format.
Who it's for
Docling Reader serves developers working with LlamaIndex who need to extract structured content from mixed document types.
Example usage
Install the reader:
pip install llama-index-readers-docling
Basic Markdown extraction:
from llama_index.readers.docling import DoclingReader
reader = DoclingReader()
docs = reader.load_data(file_path="https://arxiv.org/pdf/2408.09869")
print(f"{docs[0].text[389:442]}...")
### > ## Abstract
### >
### > This technical report introduces Docling...
JSON export for native format:
from llama_index.readers.docling import DoclingReader
reader = DoclingReader(export_type=DoclingReader.ExportType.JSON)
docs = reader.load_data(file_path="https://arxiv.org/pdf/2408.09869")
print(f"{docs[0].text[:53]}...")
### > {"schema_name": "DoclingDocument", "version": "1.0.0"...
Source README
Docling Reader
Overview
Docling Reader uses Docling to enable fast and easy extraction of PDF, DOCX, HTML, and other document types, into Markdown or JSON-serialized Docling format, for usage in LlamaIndex pipelines for RAG / QA etc.
Installation
pip install llama-index-readers-docling
Usage
Markdown export
By default, Docling Reader exports to Markdown. Basic usage looks like this:
from llama_index.readers.docling import DoclingReader
reader = DoclingReader()
docs = reader.load_data(file_path="https://arxiv.org/pdf/2408.09869")
print(f"{docs[0].text[389:442]}...")
### > ## Abstract
### >
### > This technical report introduces Docling...
JSON export
Docling Reader can also export Docling's native format to JSON:
from llama_index.readers.docling import DoclingReader
reader = DoclingReader(export_type=DoclingReader.ExportType.JSON)
docs = reader.load_data(file_path="https://arxiv.org/pdf/2408.09869")
print(f"{docs[0].text[:53]}...")
### > {"schema_name": "DoclingDocument", "version": "1.0.0"...
With Simple Directory Reader
The Docling Reader can also be used directly in combination with Simple Directory Reader, for example:
from llama_index.core import SimpleDirectoryReader
dir_reader = SimpleDirectoryReader(
input_dir="/path/to/docs",
file_extractor={".pdf": reader},
)
docs = dir_reader.load_data()
print(docs[0].metadata)
### > {'file_path': '/path/to/docs/2408.09869v3.pdf',
### > 'file_name': '2408.09869v3.pdf',
### > 'file_type': 'application/pdf',
### > 'file_size': 5566574,
### > 'creation_date': '2024-10-06',
### > 'last_modified_date': '2024-10-03'}
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.