Download and Process SEC Filings
Download and organize US SEC 10-K and 10-Q filings by ticker for LlamaIndex indexing.
Why it matters
Automate the retrieval and organization of SEC filings (10-K and 10-Q) for specified tickers. This asset facilitates data ingestion for further analysis and research.
Outcomes
What it gets done
Download 10-K and 10-Q filings from the SEC.
Organize downloaded filings by ticker and year.
Prepare SEC filing data for indexing and querying.
Support integration with LlamaIndex and Langchain for RAG applications.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/li-reader-readers-sec-filings | bash Overview
SEC DATA DOWNLOADER
A LlamaIndex loader that downloads SEC 10-K and 10-Q filings for given tickers into a year-organized directory structure. Use to get local, indexable SEC filing text organized by ticker and year for question-answering over disclosures.
What it does
The SEC Data Downloader loads all the text from SEC filings - 10-K annual reports and 10-Q quarterly reports - for a list of tickers, saving them into a structured directory layout for LlamaIndex to index. It does not currently support amended filings, though that is noted as planned. The project builds on the author's separate SEC-QA-Agent repository.
SECFilingsLoader takes five attributes: a list of tickers, the amount of documents to download per ticker, the filing_type (10-K or 10-Q), num_workers for multithreading at the ticker level and multiprocessing at the year level, and include_amends to control whether amended filings are included. Calling load_data() downloads the filings into per-ticker, per-year directories, with 10-Q filings additionally organized by the month they cover (for example 10-Q_03.json for March), and amended filings stored in their respective year folder - the source's own worked example shows this for AAPL, GOOGL, and TSLA, each getting a top-level folder with year subfolders holding one or more 10-K.json/10-Q_MM.json files, plus a 10-KA.json for TSLA's 2021 amended filing.
The README also credits two reference implementations the loader draws on: the Unstructured SEC Filings API and the SEC Edgar Downloader.
When to use - and when NOT to
Use it when you need the actual text of a company's SEC filings - risk factors, financial disclosures, management discussion - available locally and organized by ticker and year, ready to load into a LlamaIndex or LangChain index for question-answering. The source's own worked example queries "What are the risk factors of Tesla for the year 2022?" against a VectorStoreIndex built from the downloaded 10-K text, and shows an equivalent LangChain path using DirectoryLoader, VectorstoreIndexCreator, and a RetrievalQA chain over the same downloaded files. Do not use it if you need amended filings included by default, or if you need filing types beyond 10-K and 10-Q.
Capabilities
load_data downloads 10-K or 10-Q filings for a list of tickers into a year-organized directory structure, with configurable document count, worker concurrency, and amendment inclusion. The downloaded JSON files can be indexed directly with either LlamaIndex's SimpleDirectoryReader/VectorStoreIndex, or LangChain's DirectoryLoader/VectorstoreIndexCreator.
How to install
pip install llama-index-readers-sec-filings
Also requires installing the project's own dependencies via pip install -r requirements.txt.
Who it's for
Developers and analysts building question-answering systems over SEC filings - for example querying a company's stated risk factors for a given fiscal year - who want the raw filing text downloaded and organized automatically rather than scraped by hand.
Source README
SEC DATA DOWNLOADER
pip install llama-index-readers-sec-filings
Please checkout this repo that I am building on SEC Question Answering Agent SEC-QA
This repository downloads all the texts from SEC documents (10-K and 10-Q). Currently, it is not supporting documents that are amended, but that will be added in the near futures.
Install the required dependencies
python install -r requirements.txt
The SEC Downloader expects 5 attributes
- tickers: It is a list of valid tickers
- amount: Number of documents that you want to download
- filing_type: 10-K or 10-Q filing type
- num_workers: It is for multithreading and multiprocessing. We have multi-threading at the ticker level and multi-processing at the year level for a given ticker
- include_amends: To include amendments or not.
Usage
from llama_index.readers.sec_filings import SECFilingsLoader
loader = SECFilingsLoader(tickers=["TSLA"], amount=3, filing_type="10-K")
loader.load_data()
It will download the data in the following directories and sub-directories
- AAPL
- 2018
- 10-K.json
- 2019
- 10-K.json
- 2020
- 10-K.json
- 2021
- 10-K.json
- 10-Q_12.json
- 2022
- 10-K.json
- 10-Q_03.json
- 10-Q_06.json
- 10-Q_12.json
- 2023
- 10-Q_04.json
- GOOGL
- 2018
- 10-K.json
- 2019
- 10-K.json
- 2020
- 10-K.json
- 2021
- 10-K.json
- 10-Q_09.json
- 2022
- 10-K.json
- 10-Q_03.json
- 10-Q_06.json
- 10-Q_09.json
- 2023
- 10-Q_03.json
- TSLA
- 2018
- 10-K.json
- 2019
- 10-K.json
- 2020
- 10-K.json
- 2021
- 10-K.json
- 10-KA.json
- 10-Q_09.json
- 2022
- 10-K.json
- 10-Q_03.json
- 10-Q_06.json
- 10-Q_09.json
- 2023
- 10-Q_03.json
Here for each ticker we have separate folders with 10-K data inside respective years and 10-Q data is saved in the respective year along with the month. 10-Q_03.json means March data of 10-Q document. Also, the amended documents are stored in their respective year
EXAMPLES
This loader is can be used with both Langchain and LlamaIndex.
LlamaIndex
from llama_index.core import VectorStoreIndex, download_loader
from llama_index.core import SimpleDirectoryReader
from llama_index.readers.sec_filings import SECFilingsLoader
loader = SECFilingsLoader(tickers=["TSLA"], amount=3, filing_type="10-K")
loader.load_data()
documents = SimpleDirectoryReader("data\TSLA\2022").load_data()
index = VectorStoreIndex.from_documents(documents)
index.query("What are the risk factors of Tesla for the year 2022?")
Langchain
from langchain.llms import OpenAI
from langchain.chains import RetrievalQA
from langchain.document_loaders import DirectoryLoader
from langchain.indexes import VectorstoreIndexCreator
from llama_index.readers.sec_filings import SECFilingsLoader
loader = SECFilingsLoader(tickers=["TSLA"], amount=3, filing_type="10-K")
loader.load_data()
dir_loader = DirectoryLoader("data\TSLA\2022")
index = VectorstoreIndexCreator().from_loaders([dir_loader])
retriever = index.vectorstore.as_retriever()
qa = RetrievalQA.from_chain_type(
llm=OpenAI(), chain_type="stuff", retriever=retriever
)
query = "What are the risk factors of Tesla for the year 2022?"
qa.run(query)
REFERENCES
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.