Load Elasticsearch Data into LlamaIndex
Elasticsearch Reader for LlamaIndex loads documents from Elasticsearch or Opensearch indexes via REST API, enabling downstream data ingestion and RAG workflows.
Why it matters
Integrate Elasticsearch or OpenSearch data into LlamaIndex for advanced data processing and retrieval. This asset enables seamless data loading from your search index into LlamaIndex structures.
Outcomes
What it gets done
Connect to Elasticsearch/OpenSearch via REST API.
Retrieve documents from specified indices.
Extract text from designated fields within documents.
Optionally load embedding fields for vector search.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/li-reader-readers-elasticsearch | bash Capabilities
What this skill does
Writes and executes SQL or NoSQL queries on databases.
Pulls structured data fields from unstructured text.
Chunks, embeds, and indexes documents for semantic retrieval.
Overview
LlamaIndex Readers Integration: Elasticsearch
What it does
This reader connects LlamaIndex to Elasticsearch and Opensearch clusters over REST API. It executes Elasticsearch JSON query DSL searches, retrieves documents from specified indexes and fields, and converts them into LlamaIndex Document objects. It supports optional embedding field extraction and configurable HTTP client parameters for timeout and connection management.
How it connects
Use this reader when you need to ingest documents stored in Elasticsearch or Opensearch into LlamaIndex data structures for retrieval-augmented generation, semantic search, or agent workflows. It's ideal for organizations with existing Elasticsearch deployments who want to leverage that data in LlamaIndex or LangChain applications without migrating storage layers.
Source README
LlamaIndex Readers Integration: Elasticsearch
Overview
Elasticsearch (or Opensearch) Reader over REST API is a tool designed to read documents from an Elasticsearch or Opensearch index using the basic search API. These documents can then be utilized in downstream LlamaIndex data structures.
Installation
You can install Elasticsearch (or Opensearch) Reader via pip:
pip install llama-index-readers-elasticsearch
Usage
from llama_index.core.schema import Document
from llama_index.readers.elasticsearch import ElasticsearchReader
# Initialize ElasticsearchReader
reader = ElasticsearchReader(
endpoint="<Your Elasticsearch/Opensearch Endpoint>",
index="<Index Name>",
httpx_client_args={
"timeout": 10
}, # Optional additional arguments for the httpx.Client
)
# Load data from Elasticsearch
documents = reader.load_data(
field="<Field Name>", # Field in the document to retrieve text from
query={"query": {"match_all": {}}}, # Elasticsearch JSON query DSL object
embedding_field="<Embedding Field>", # Field for embeddings (optional)
)
This loader is designed to be used as a way to load data into
LlamaIndex and/or subsequently
used as a Tool in a LangChain Agent.
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.