Build Hierarchical Knowledge Graphs for Advanced Retrieval
Two LlamaIndex packs for hierarchical small-to-big retrieval: one for embedded tables via Unstructured.io, one general-purpose recursive retriever.
Maintainer of this project? Claim this page to edit the listing.
0.14.23Add to Favorites
Why it matters
Leverage advanced retrieval techniques to build hierarchical node graphs from documents, enabling more nuanced and effective querying of complex information.
Outcomes
What it gets done
Load and parse documents into a hierarchical structure.
Implement recursive retrieval for small-to-big information access.
Utilize Unstructured.io for robust document parsing, including tables.
Provide query engine for efficient information retrieval from the knowledge graph.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/li-pack-packs-recursive-retriever | bash Overview
Recursive Retriever Packs
Two LlamaIndex packs implementing small-to-big hierarchical retrieval: one built for documents with embedded tables via Unstructured.io parsing, one general-purpose for any document collection, both downloadable as editable code templates. Use the table-focused pack for documents with structured tables (financial filings, reports); use the general pack for other document collections needing the same hierarchical retrieval pattern. Both are editable templates, not opaque libraries.
What it does
Provides two related LlamaPacks implementing recursive, small-to-big retrieval, where a document is parsed into a hierarchical node graph of bigger parent nodes and smaller child nodes, so retrieval can match on precise small chunks while returning the richer parent context around them.
The first, EmbeddedTablesUnstructuredRetrieverPack, is specifically built for documents containing embedded tables, using Unstructured.io to parse the source document (the example uses a Tesla 2021 10-K HTML filing) and build the hierarchical node graph around both prose and table content. It's installable via the LlamaIndex CLI (llamaindex-cli download-llamapack EmbeddedTablesUnstructuredRetrieverPack --download-dir ./embedded_tables_unstructured_pack) or programmatically via download_llama_pack, which downloads the pack code and its dependencies into a local directory that can be inspected or modified like any other project template.
embedded_tables_unstructured_pack = EmbeddedTablesUnstructuredRetrieverPack(
"tesla_2021_10k.htm",
)
response = embedded_tables_unstructured_pack.run("What was the revenue in 2020?")
The second, RecursiveRetrieverSmallToBigPack, is the general-purpose version of the same small-to-big retrieval pattern, built from any document set rather than being table-specific. It follows the identical installation and usage shape - CLI download to ./recursive_retriever_stb_pack, or download_llama_pack("RecursiveRetrieverSmallToBigPack", ...) - constructed from a list of documents rather than a single filename, and queried the same way via run().
Both packs expose their internal components for direct use rather than only the packaged run() convenience method: node_parser (the component that builds the hierarchical parent/child node graph), recursive_retriever (the retriever that navigates from small to big nodes), and query_engine (the assembled query interface wrapping the retriever). run() itself is documented as "a light wrapper around query_engine.query()," so either the shortcut or the underlying query engine can be used depending on how much control is needed.
When to use - and when NOT to
Use EmbeddedTablesUnstructuredRetrieverPack when the source documents contain tables that need to be parsed and retrieved accurately (financial filings, reports with structured data) and Unstructured.io's table-aware parsing is a good fit. Use RecursiveRetrieverSmallToBigPack for the same hierarchical small-to-big retrieval pattern on general document collections without a specific table-parsing need. Both are LlamaPacks, meaning they're downloaded as editable templates rather than installed as opaque dependencies - use them when you want a working starting point to inspect and customize, not a black-box library call.
Inputs and outputs
Input is a source document (or documents) plus a natural-language query. Output is a query engine response generated from the hierarchical small-to-big retrieval process, drawing on the appropriate parent-node context around the best-matching child nodes.
Integrations
Built on the LlamaIndex llama_pack download mechanism (download_llama_pack or llamaindex-cli download-llamapack), with the table-focused pack additionally depending on Unstructured.io for document parsing.
Who it's for
Developers building retrieval-augmented applications who need hierarchical small-to-big retrieval - either specifically for documents with embedded tables (financial filings, structured reports) or for general document collections - and want an editable LlamaPack template rather than building the node-graph and retriever wiring from scratch.
Source README
Recursive Retriever Packs
Embedded Tables Retriever Pack w/ Unstructured.io
This LlamaPack provides an example of our embedded tables retriever.
This specific template shows the e2e process of building this. It loads
a document, builds a hierarchical node graph (with bigger parent nodes and smaller
child nodes).
Check out the notebook here.
CLI Usage
You can download llamapacks directly using llamaindex-cli, which comes installed with the llama-index python package:
llamaindex-cli download-llamapack EmbeddedTablesUnstructuredRetrieverPack --download-dir ./embedded_tables_unstructured_pack
You can then inspect the files at ./embedded_tables_unstructured_pack and use them as a template for your own project.
Code Usage
You can download the pack to a the ./embedded_tables_unstructured_pack directory:
from llama_index.core.llama_pack import download_llama_pack
### download and install dependencies
EmbeddedTablesUnstructuredRetrieverPack = download_llama_pack(
"EmbeddedTablesUnstructuredRetrieverPack",
"./embedded_tables_unstructured_pack",
)
From here, you can use the pack, or inspect and modify the pack in ./embedded_tables_unstructured_pack.
Then, you can set up the pack like so:
### create the pack
### get documents from any data loader
embedded_tables_unstructured_pack = EmbeddedTablesUnstructuredRetrieverPack(
"tesla_2021_10k.htm",
)
The run() function is a light wrapper around query_engine.query().
response = embedded_tables_unstructured_pack.run(
"What was the revenue in 2020?"
)
You can also use modules individually.
### get the node parser
node_parser = embedded_tables_unstructured_pack.node_parser
### get the retriever
retriever = embedded_tables_unstructured_pack.recursive_retriever
### get the query engine
query_engine = embedded_tables_unstructured_pack.query_engine
Recursive Retriever - Small-to-big retrieval
This LlamaPack provides an example of our recursive retriever (small-to-big).
This specific template shows the e2e process of building this. It loads
a document, builds a hierarchical node graph (with bigger parent nodes and smaller
child nodes).
Check out the notebook here.
CLI Usage
You can download llamapacks directly using llamaindex-cli, which comes installed with the llama-index python package:
llamaindex-cli download-llamapack RecursiveRetrieverSmallToBigPack --download-dir ./recursive_retriever_stb_pack
You can then inspect the files at ./recursive_retriever_stb_pack and use them as a template for your own project.
Code Usage
You can download the pack to a the ./recursive_retriever_stb_pack directory:
from llama_index.core.llama_pack import download_llama_pack
### download and install dependencies
RecursiveRetrieverSmallToBigPack = download_llama_pack(
"RecursiveRetrieverSmallToBigPack", "./recursive_retriever_stb_pack"
)
From here, you can use the pack, or inspect and modify the pack in ./recursive_retriever_stb_pack.
Then, you can set up the pack like so:
### create the pack
### get documents from any data loader
recursive_retriever_stb_pack = RecursiveRetrieverSmallToBigPack(
documents,
)
The run() function is a light wrapper around query_engine.query().
response = recursive_retriever_stb_pack.run(
"Tell me a bout a Music celebrity."
)
You can also use modules individually.
### get the recursive retriever
recursive_retriever = recursive_retriever_stb_pack.recursive_retriever
### get the query engine
query_engine = recursive_retriever_stb_pack.query_engine
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.