Load Zendesk Support Data with Airbyte
Load Zendesk Support objects like tickets into LlamaIndex via Airbyte, with incremental sync.
Why it matters
Effortlessly extract and load your Zendesk Support data into your preferred data pipelines. This asset enables seamless integration with LlamaIndex for further analysis and AI-powered applications.
Outcomes
What it gets done
Connect to Zendesk Support via Airbyte.
Extract various Zendesk Support objects (e.g., tickets).
Load data incrementally or in full.
Prepare data for use with LlamaIndex.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/li-reader-readers-airbyte-zendesk-support | bash Overview
Airbyte ZendeskSupport Loader
A LlamaIndex reader that loads Zendesk Support data via Airbyte's Zendesk connector, scoped by subdomain and start date, with lazy and incremental loading. Use for Zendesk Support data needing date-bounded retrieval, with lazy loading for volume and incremental sync for recurring runs.
What it does
The Airbyte ZendeskSupport Loader gives access to different Zendesk Support objects through the Airbyte Zendesk Support connector, letting LlamaIndex load data such as tickets without a custom integration. AirbyteZendeskSupportReader is initialized with a config dictionary matching Airbyte's Zendesk Support source spec, and load_data takes a stream_name (such as tickets) identifying which object stream to pull.
The config object follows Airbyte's own documented JSON schema and includes a subdomain identifying the Zendesk account, a start_date in ISO format bounding how far back records are retrieved, and credentials carrying an API-token-based auth (a credentials type, an email, and an API token).
By default, every field of a record is stored as document metadata and the document text is the JSON representation of all fields; a custom record_handler function can be passed to construct the text and metadata differently, for example using just a record's title field as the document text. Beyond the standard load_data, lazy_load_data returns an iterator instead of a full list for streaming large record sets without holding everything in memory, and incremental loading lets the reader track state (reader.last_state, picklable for storage) so a later load_data call with that state returns only records that are new or updated since the last run.
When to use - and when NOT to
Use it when you need Zendesk Support data - tickets and other support objects - loaded into LlamaIndex, scoped by a start date for a specific Zendesk subdomain. Use lazy_load_data for large ticket volumes to avoid memory pressure, and incremental loading for recurring syncs where only new or changed records matter. Do not omit start_date; the config schema requires it to bound the retrieval window.
Capabilities
load_data/lazy_load_data pull a named Zendesk Support stream (such as tickets) scoped by start date, with default JSON-as-text document construction, an optional custom record_handler, and incremental loading via a trackable last_state.
How to install
pip install llama-index-readers-airbyte-zendesk-support
Requires a Zendesk config object with a subdomain, a start_date, and API-token credentials (email and API token).
Who it's for
Developers who need Zendesk Support tickets and related data loaded into LlamaIndex, especially for large volumes (via lazy loading) or recurring incremental syncs.
Source README
Airbyte ZendeskSupport Loader
pip install llama-index-readers-airbyte-zendesk-support
The Airbyte ZendeskSupport Loader allows you to access different ZendeskSupport objects.
Usage
Here's an example usage of the AirbyteZendeskSupportReader.
from llama_index.readers.airbyte_zendesk_support import (
AirbyteZendeskSupportReader,
)
zendesk_support_config = {
# ...
}
reader = AirbyteZendeskSupportReader(config=zendesk_support_config)
documents = reader.load_data(stream_name="tickets")
Configuration
Check out the Airbyte documentation page for details about how to configure the reader.
The JSON schema the config object should adhere to can be found on Github: https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-zendesk-support/source_zendesk_support/spec.json.
The general shape looks like this:
{
"subdomain": "<your zendesk subdomain>",
"start_date": "<date from which to start retrieving records from in ISO format, e.g. 2020-10-20T00:00:00Z>",
"credentials": {
"credentials": "api_token",
"email": "<your email>",
"api_token": "<your api token>",
},
}
By default all fields are stored as metadata in the documents and the text is set to the JSON representation of all the fields. Construct the text of the document by passing a record_handler to the reader:
def handle_record(record, id):
return Document(
doc_id=id, text=record.data["title"], extra_info=record.data
)
reader = AirbyteZendeskSupportReader(
config=zendesk_support_config, record_handler=handle_record
)
Lazy loads
The reader.load_data endpoint will collect all documents and return them as a list. If there are a large number of documents, this can cause issues. By using reader.lazy_load_data instead, an iterator is returned which can be consumed document by document without the need to keep all documents in memory.
Incremental loads
This loader supports loading data incrementally (only returning documents that weren't loaded last time or got updated in the meantime):
reader = AirbyteZendeskSupportReader(config={...})
documents = reader.load_data(stream_name="tickets")
current_state = reader.last_state # can be pickled away or stored otherwise
updated_documents = reader.load_data(
stream_name="tickets", state=current_state
) # only loads documents that were updated since last time
This loader is designed to be used as a way to load data into LlamaIndex.
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.