Load Stripe Documentation for AI Analysis
Crawl and index Stripe's documentation site into LlamaIndex for Q&A over Stripe docs.
Why it matters
Ingest and index the entire Stripe documentation library to build a knowledge base for AI-powered querying and analysis of Stripe's services and APIs.
Outcomes
What it gets done
Asynchronously scrape Stripe's official documentation pages.
Extract relevant text content from HTML documentation.
Index the scraped documentation for efficient retrieval.
Filter documentation based on specific paths or keywords.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/li-reader-readers-stripe-docs | bash Overview
StripeDocs Loader
A LlamaIndex loader that asynchronously crawls Stripe's documentation sitemap into documents, with plain-text output and section filtering. Use to build Q&A over Stripe's documentation, scoped to a specific section via filters if needed.
What it does
The StripeDocs Loader asynchronously loads data from Stripe's official documentation site, iterating through the Stripe sitemap to find all /docs references and loading their content as LlamaIndex documents. It is built on top of the Async Website Loader, reusing that asynchronous crawling approach rather than implementing its own from scratch.
StripeDocsReader is instantiated with no required arguments, and load_data() crawls and loads the documentation pages. Two options shape how it behaves: html_to_text=True returns plain text instead of raw HTML, and limit (for example limit=10) caps how many concurrent requests are made against the Stripe site. A filters argument passed to load_data controls which pages from the Stripe sitemap get loaded - the default filter is ["/docs"], scoping everything to documentation pages only, but it can be overridden (for example to ["/terminal"]) to load a different section of the Stripe site instead.
When to use - and when NOT to
Use it when you want to build a question-answering system grounded in Stripe's actual documentation - the source's own example indexes the loaded docs and queries "How do I accept payments on my website?" directly against them. Use filters to scope the crawl to a specific Stripe documentation section (like /terminal) rather than the entire docs site if you only need answers about one product area. Do not use it for content outside Stripe's own site; the crawler is scoped to the Stripe sitemap.
Capabilities
load_data asynchronously crawls Stripe's documentation sitemap and loads matching pages as documents, with html_to_text for plain-text output, limit for concurrency control, and filters to scope which sitemap sections get loaded.
How to install
pip install llama-index-readers-stripe-docs
Who it's for
Developers building a Q&A or search system over Stripe's documentation, who want the crawl scoped to a specific product area rather than the entire docs site.
Source README
StripeDocs Loader
pip install llama-index-readers-stripe-docs
This loader asynchronously loads data from the Stripe documentation. It iterates through the Stripe sitemap to get all /docs references.
It is based on the Async Website Loader.
Usage
from llama_index.core import VectorStoreIndex
from llama_index.readers.stripe_docs import StripeDocsReader
loader = StripeDocsReader()
documents = loader.load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
query_engine.query("How do I accept payments on my website?")
The StripeDocsReader allows you to return plain text docs by setting html_to_text=True. You can also adjust the maximum concurrent requests by setting limit=10.
Filtering
You can filter pages from the Stripe sitemap by adding the filters argument to the load_data method. This allows you to control what pages from the Stripe website, including documentation, will be loaded.
The default filters are set to ["/docs"] to scope everything to docs only.
documents = loader.load_data(filters=["/terminal"])
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.