Connect to WordLift Knowledge Graph
WordLift Reader fetches data from a WordLift Knowledge Graph using GraphQL queries and transforms it into a list of documents for further processing.
Why it matters
Integrate your applications with the WordLift Knowledge Graph to extract and process structured data. Leverage your WordLift Key to query and transform data into documents for further analysis and AI processing.
Outcomes
What it gets done
Fetch data from WordLift using GraphQL queries.
Transform raw data into a list of processable documents.
Configure API endpoints, headers, and query parameters.
Prepare data for indexing and querying with AI models.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/li-reader-readers-wordlift | bash Overview
WordLift Reader
A LlamaIndex reader that runs a GraphQL query against a WordLift Knowledge Graph and maps the results to document text and metadata. Use when content is already modeled in a WordLift Knowledge Graph and you have a GraphQL query tailored to its schema.
What it does
The WordLift GraphQL Reader connects to a WordLift Knowledge Graph and fetches data using a GraphQL query, transforming the results into a list of LlamaIndex documents for further processing. Authentication runs through a WordLift Key, passed as an Authorization header alongside the GraphQL endpoint URL.
WordLiftLoader is initialized with an endpoint (the WordLift GraphQL API URL), headers (carrying the WordLift Key and content type), a query (the actual GraphQL query string), fields, and a config_options dictionary specifying text_fields (which returned fields become document text) and metadata_fields (which become document metadata). load_data() then runs the configured query and returns the resulting documents.
The source's own worked example goes a step further than a bare load: it takes the documents returned by load_data() and rebuilds each one as a plain LlamaIndex Document - re-serializing the doc_id through json.dumps, and copying over text, embedding, doc_hash, and extra_info - before building a VectorStoreIndex from the converted documents and running a query against it via index.as_query_engine().
When to use - and when NOT to
Use it when you need to pull data out of a WordLift Knowledge Graph via GraphQL and index it in LlamaIndex - for example building a query engine over semantically structured content already modeled in WordLift. Configure text_fields and metadata_fields deliberately, since they control which parts of each GraphQL result become searchable text versus filterable metadata. Do not use it without a valid WordLift Key and a working GraphQL query already tailored to your WordLift Knowledge Graph's schema; the reader executes whatever query it's given rather than discovering the schema for you.
Capabilities
load_data runs a configured GraphQL query against a WordLift Knowledge Graph and returns documents, with text_fields/metadata_fields controlling how each result's fields map onto document text versus metadata.
How to install
pip install llama-index-readers-wordlift
Requires a WordLift Key and a GraphQL query tailored to the target WordLift Knowledge Graph's schema.
Who it's for
Developers who already model content in a WordLift Knowledge Graph and want a GraphQL-queried subset of it loaded into LlamaIndex for indexing and retrieval.
Source README
WordLift Reader
pip install llama-index-readers-wordlift
The WordLift GraphQL Reader is a connector to fetch and transform data from a WordLift Knowledge Graph using your the WordLift Key. The connector provides a convenient way to load data from WordLift using a GraphQL query and transform it into a list of documents for further processing.
Usage
To use the WordLift GraphQL Reader, follow the steps below:
- Set up the necessary configuration options, such as the API endpoint, headers, query, fields, and configuration options (make sure you have with you the Wordlift Key).
- Create an instance of the
WordLiftLoaderclass, passing in the configuration options. - Use the
load_datamethod to fetch and transform the data. - Process the loaded documents as needed.
Here's an example of how to use the WordLift GraphQL Reader:
import json
from llama_index.core import VectorStoreIndex
from llama_index.core import Document
from langchain.llms import OpenAI
from llama_index.readers.wordlift import WordLiftLoader
### Set up the necessary configuration options
endpoint = "https://api.wordlift.io/graphql"
headers = {
"Authorization": "<YOUR_WORDLIFT_KEY>",
"Content-Type": "application/json",
}
query = """
### Your GraphQL query here
"""
fields = "<YOUR_FIELDS>"
config_options = {
"text_fields": ["<YOUR_TEXT_FIELDS>"],
"metadata_fields": ["<YOUR_METADATA_FIELDS>"],
}
### Create an instance of the WordLiftLoader
reader = WordLiftLoader(endpoint, headers, query, fields, config_options)
### Load the data
documents = reader.load_data()
### Convert the documents
converted_doc = []
for doc in documents:
converted_doc_id = json.dumps(doc.doc_id)
converted_doc.append(
Document(
text=doc.text,
doc_id=converted_doc_id,
embedding=doc.embedding,
doc_hash=doc.doc_hash,
extra_info=doc.extra_info,
)
)
### Create the index and query engine
index = VectorStoreIndex.from_documents(converted_doc)
query_engine = index.as_query_engine()
### Perform a query
result = query_engine.query("<YOUR_QUERY>")
### Process the result as needed
logging.info("Result: %s", result)
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.