Fetch and Index Readwise Highlights
Readwise Reader loader fetches highlights from web articles, PDFs, Kindle, YouTube, and ePubs into LlamaIndex for LLM-powered search and retrieval.
Why it matters
Ingest your personal knowledge base from Readwise, including highlights from articles, PDFs, and Kindle, to power LLM-based querying and analysis.
Outcomes
What it gets done
Connect to the Readwise API using your access token.
Retrieve highlights from various sources like web articles, epubs, and PDFs.
Load extracted text into LlamaIndex for further processing.
Optionally filter highlights based on update time.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/li-reader-readers-readwise | bash Overview
Readwise Reader
Readwise Reader is a LlamaIndex data loader that fetches your highlights from web articles, ePubs, PDFs, Kindle, and YouTube using Readwise's export API. It loads the text into LlamaIndex vector stores, enabling natural language queries across your reading history. Use Readwise Reader when you want to build a searchable knowledge base from your Readwise highlights. It's ideal for querying recent highlights or searching your entire reading library with time-filtered retrieval options.
What it does
Readwise Reader is a LlamaIndex data loader that uses Readwise's export API to fetch your highlights from web articles, ePubs, PDFs, Kindle, and YouTube. It loads the resulting text into LlamaIndex vector stores, enabling you to query your reading highlights with LLMs.
When to use - and when NOT to
Use Readwise Reader when you want to build a searchable knowledge base from your Readwise highlights across multiple content sources. It's ideal for querying recent highlights (e.g., "What has Elon Musk done this time?") or searching your entire reading history (e.g., "What was the paper 'Attention is all you need' about?"). The loader supports time-filtered queries, so you can retrieve only highlights created after a specific date.
Do NOT use this loader if you don't have a Readwise account or if you need real-time access to content before it's been highlighted and synced to Readwise. This tool retrieves only your saved highlights, not the full text of every document you've read.
Inputs and outputs
You provide a Readwise API key (obtained from readwise.io/access_token) and optionally an updated_after datetime parameter to filter highlights by creation date. The loader returns LlamaIndex Document objects containing your highlight text, which can be indexed and queried.
Integrations
Readwise Reader integrates with LlamaIndex's VectorStoreIndex for document indexing and retrieval. It pulls data from Readwise's export API, which aggregates highlights from web articles, ePubs, PDFs, Kindle, and YouTube.
Who it's for
This loader is for researchers, knowledge workers, and avid readers who use Readwise to capture highlights and want to query their reading library using natural language. It's particularly useful for anyone building a personal knowledge management system with LlamaIndex.
Example usage
Install the package:
pip install llama-index-readers-readwise
Basic usage to query all highlights:
import os
from llama_index.core import VectorStoreIndex, download_loader
from llama_index.readers.readwise import ReadwiseReader
token = os.getenv("READWISE_API_KEY")
loader = ReadwiseReader(api_key=token)
documents = loader.load_data()
index = VectorStoreIndex.from_documents(documents)
index.query("What was the paper 'Attention is all you need' about?")
Query highlights from the last seven days:
import os
import datetime
from llama_index.core import VectorStoreIndex, download_loader
from llama_index.readers.readwise import ReadwiseReader
token = os.getenv("READWISE_API_KEY")
loader = ReadwiseReader(api_key=token)
seven_days_ago = datetime.datetime.now() - datetime.timedelta(days=7)
documents = loader.load_data(updated_after=seven_days_ago)
index = VectorStoreIndex.from_documents(documents)
index.query("What has Elon Musk done this time?")
Source README
Readwise Reader
pip install llama-index-readers-readwise
Use Readwise's export API to fetch your highlights from web articles, epubs, pdfs, Kindle, YouTube, and load the resulting text into LLMs.
Setup
- Get your Readwise API key from readwise.io/access_token.
Usage
Here is an example usage of the Readwise Reader:
import os
from llama_index.core import VectorStoreIndex, download_loader
from llama_index.readers.readwise import ReadwiseReader
token = os.getenv("READWISE_API_KEY")
loader = ReadwiseReader(api_key=token)
documents = loader.load_data()
index = VectorStoreIndex.from_documents(documents)
index.query("What was the paper 'Attention is all you need' about?")
You can also query for highlights that have been created after a certain time:
import os
import datetime
from llama_index.core import VectorStoreIndex, download_loader
from llama_index.readers.readwise import ReadwiseReader
token = os.getenv("READWISE_API_KEY")
loader = ReadwiseReader(api_key=token)
seven_days_ago = datetime.datetime.now() - datetime.timedelta(days=7)
documents = loader.load_data(updated_after=seven_days_ago)
index = VectorStoreIndex.from_documents(documents)
index.query("What has Elon Musk done this time?")
This loader is designed to be used as a way to load data into LlamaIndex.
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.