Load Steamship Files into LlamaIndex

The SteamshipFile Reader is a LlamaIndex integration that loads documents from persistent Steamship Files. Steamship is a platform for storing and managing files with advanced tagging capabilities, and this reader provides a bridge to ingest those files into LlamaIndex workflows. It enables querying files by workspace, tag queries, or specific file handles, with options to collapse blocks and control text joining.

Use this reader when you have documents already stored in Steamship and need to bring them into LlamaIndex for indexing, retrieval, or agent workflows. Use it when you need persistent file storage with metadata that can be queried programmatically.

Do not use this reader if your documents are not stored in Steamship - you would need a different LlamaIndex reader for local files, cloud storage, or other platforms.

You provide a Steamship API key for authentication, a workspace identifier, and either a tag query or specific file handles to target documents. Optional parameters include collapse_blocks (boolean) to control block merging and join_str to specify how text segments are concatenated.

You receive documents loaded from the matching Steamship Files.

Install via pip:

pip install llama-index-readers-steamship

Basic usage example:

from llama_index.readers.steamship import SteamshipFileReader

### Initialize SteamshipFileReader
reader = SteamshipFileReader(api_key="<Steamship API Key>")

### Load data from persistent Steamship Files
documents = reader.load_data(
    workspace="<Steamship Workspace>",
    query="<Steamship Tag Query>",
    file_handles=["smooth-valley-9kbdr"],
    collapse_blocks=True,
    join_str="\n\n",
)

LlamaIndex: The reader is designed as a data loader for LlamaIndex, integrating directly with its document ingestion pipeline.

LangChain: Can be used as a Tool in LangChain Agent workflows after loading data into LlamaIndex.

Steamship: Requires a Steamship API key and connects to Steamship workspaces to access persistent files with tagging capabilities.

This reader is for developers building LlamaIndex applications who need to load documents from Steamship. Data engineers and AI application developers who need to bridge Steamship's persistent storage with LlamaIndex's indexing and querying capabilities will find this integration useful.