Load Firestore Data into LlamaIndex
Firestore Loader reads documents and collections from Google Cloud Firestore into LlamaIndex Document objects.
Why it matters
Effortlessly ingest data from your Google Cloud Firestore database into LlamaIndex for advanced AI applications. This loader handles both collections and individual documents, streamlining your data pipeline.
Outcomes
What it gets done
Load data from Firestore collections, including nested paths.
Retrieve specific documents from Firestore.
Prepare Firestore data for use with LlamaIndex.
Integrate with Google Cloud authentication for secure access.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/li-reader-readers-firestore | bash Overview
Firestore Loader
Firestore Loader extracts documents and collections from Google Cloud Firestore and converts them into LlamaIndex Document objects. It supports both individual document retrieval and full collection loading, including nested collection paths. The loader assumes Google Cloud credentials are already configured via Application Default Credentials. Use Firestore Loader when you need to load data from Google Cloud Firestore into LlamaIndex. It is designed for developers who already have data in Firestore and want to convert it into LlamaIndex Document objects.
What it does
Firestore Loader is a data reader that extracts documents and collections from Google Cloud Firestore and converts them into LlamaIndex Document objects. It enables developers to ingest Firestore data into LlamaIndex.
When to use - and when NOT to
Use Firestore Loader when you need to load data from Firestore into LlamaIndex. It handles both top-level and nested collection paths.
Do not use this loader if your Google Cloud credentials are not already configured via Application Default Credentials (ADC), as the loader assumes credentials are present. Do not use it for real-time streaming; it performs point-in-time data loads rather than listening to Firestore changes.
Inputs and outputs
You provide a Google Cloud project ID during initialization and either a collection path or a specific document URL when loading data. Collection paths must include all parent documents and collections for nested structures (e.g., "foo/bar/abc/").
The load_data method returns a list of Document objects from an entire collection, while the load_document method returns a single Document object for a specific document.
Integrations
Firestore Loader integrates with Google Cloud Firestore as its data source and LlamaIndex as its target framework for document ingestion and indexing.
Who it's for
This loader is designed for developers who need to load data from Google Cloud Firestore into LlamaIndex.
Installation and usage
Install the package:
pip install llama-index-readers-firestore
Initialize the reader with your project ID:
from llama_index.readers.firestore import FirestoreReader
reader = FirestoreReader(project_id="<Your Project ID>")
Load an entire collection:
documents = reader.load_data(collection="foo/bar/abc/")
Load a single document:
document = reader.load_document(document_url="foo/bar/abc/MY_DOCUMENT")
Source README
Firestore Loader
pip install llama-index-readers-firestore
This loader loads from a Firestore collection or a specific document from Firestore. The loader assumes your project already has the google cloud credentials loaded. To find out how to set up credentials, see here.
Usage
To initialize the loader, provide the project-id of the google cloud project.
Initializing the reader
from llama_index.readers.firestore import FirestoreReader
reader = FirestoreReader(project_id="<Your Project ID>")
Loading Data from a Firestore Collection
Load data from a Firestore collection with the load_data method:
The collection path should include all previous documents and collections if it is a nested collection.
documents = reader.load_data(collection="foo/bar/abc/")
Loading a Single Document from Firestore
Load a single document from Firestore with the load_document method:
document = reader.load_document(document_url="foo/bar/abc/MY_DOCUMENT")
Note: load_data returns a list of Document objects, whereas load_document returns a single Document object.
This loader is designed to be used as a way to load data into LlamaIndex.
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.