Load Data from DeepLake with LlamaIndex
LlamaIndex reader that retrieves documents from DeepLake datasets using vector similarity search with configurable distance metrics and result limits.
Why it matters
Efficiently retrieve documents from DeepLake datasets for use within LlamaIndex or as a tool in LangChain agents.
Outcomes
What it gets done
Connect to and query DeepLake datasets.
Load retrieved data into LlamaIndex Document objects.
Integrate DeepLake data retrieval into LangChain agents.
Specify query vectors, dataset paths, and retrieval limits.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/li-reader-readers-deeplake | bash Capabilities
What this skill does
Pulls structured data fields from unstructured text.
Chunks, embeds, and indexes documents for semantic retrieval.
Writes and executes SQL or NoSQL queries on databases.
Overview
LlamaIndex Readers Integration: Deeplake
What it does
DeepLake Reader is a LlamaIndex integration that retrieves documents from existing DeepLake datasets using vector similarity search. It accepts query vectors, dataset paths, result limits, and distance metrics (like L2) to fetch relevant documents. The reader requires a DeepLake API token for authentication and returns documents in LlamaIndex schema format.
How it connects
Use this reader when you have documents stored in DeepLake datasets and need to query them within LlamaIndex pipelines or LangChain agents. It's ideal for retrieval-augmented generation workflows where you want to leverage existing DeepLake vector stores without migrating data to a different system.
Source README
LlamaIndex Readers Integration: Deeplake
Overview
DeepLake Reader is a tool designed to retrieve documents from existing DeepLake datasets efficiently.
Installation
You can install DeepLake Reader via pip:
pip install llama-index-readers-deeplake
To use Deeplake Reader, you must have an API key. Here are the installation instructions
Usage
from llama_index.core.schema import Document
from llama_index.readers.deeplake import DeepLakeReader
# Initialize DeepLakeReader with the token
reader = DeepLakeReader(token="<Your DeepLake Token>")
# Load data from DeepLake
documents = reader.load_data(
query_vector=[0.1, 0.2, 0.3], # Query vector
dataset_path="<Path to Dataset>", # Path to the DeepLake dataset
limit=4, # Number of results to return
distance_metric="l2", # Distance metric
)
This loader is designed to be used as a way to load data into
LlamaIndex and/or subsequently
used as a Tool in a LangChain Agent.
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.