Retrieval Augmented Generation with a Graph Database

Retrieval Augmented Generation with a Graph Database

This notebook shows how to use LLMs in combination with Neo4j, a graph database, to perform Retrieval Augmented Generation (RAG).

Why use RAG?

If you want to use LLMs to generate answers based on your own content or knowledge base, instead of providing large context when prompting the model, you can fetch the relevant information in a database and use this information to generate a response.

This allows you to:

• Reduce hallucinations
• Provide relevant, up to date information to your users
• Leverage your own content/knowledge base

Why use a graph database?

If you have data where relationships between data points are important and you might want to leverage that, then it might be worth considering graph databases instead of traditional relational databases.

Graph databases are good to address the following:

• Navigating deep hierarchies
• Finding hidden connections between items
• Discovering relationships between items

Use cases

Graph databases are particularly relevant for recommendation systems, network relationships or analysing correlation between data points.

Example use cases for RAG with graph databases include:

• Recommendation chatbot
• AI-augmented CRM
• Tool to analyse customer behavior with natural language

Depending on your use case, you can assess whether using a graph database makes sense.

In this notebook, we will build a product recommendation chatbot, with a graph database that contains Amazon products data.

Setup

We will start by installing and importing the relevant libraries.

Make sure you have your OpenAI account set up and you have your OpenAI API key handy.

Dataset

We will use a dataset that was created from a relational database and converted to a json format, creating relationships between entities with the completions API.

We will then load this data into the graph db to be able to query it.

Loading dataset

Connecting to db

Importing data

Querying the database

Creating vector indexes

In order to efficiently search our database for terms closely related to user queries, we need to use embeddings. To do this, we will create vector indexes on each type of property.

We will be using the OpenAIEmbeddings Langchain utility. It's important to note that Langchain adds a pre-processing step, so the embeddings will slightly differ from those generated directly with the OpenAI embeddings API.

Querying the database directly

Using GraphCypherQAChain, we can generate queries against the database using Natural Language.

Extracting entities from the prompt

However, there is little added value here compared to just writing the Cypher queries ourselves, and it is prone to error.

Indeed, asking an LLM to generate a Cypher query directly might result in the wrong parameters being used, whether it's the entity type or the relationship type, as is the case above.

We will instead use LLMs to decide what to search for, and then generate the corresponding Cypher queries using templates.

For this purpose, we will instruct our model to find relevant entities in the user prompt that can be used to query our database.

Generating queries

Now that we know what to look for, we can generate the corresponding Cypher queries to query our database.

However, the entities extracted might not be an exact match with the data we have, so we will use the GDS cosine similarity function to return products that have relationships with entities similar to what the user is asking.

Finding similar items

We can then leverage the graph db to find similar products based on common characteristics.

This is where the use of a graph db really comes into play.

For example, we can look for products that are the same category and have another characteristic in common, or find products that have relationships to the same entities.

This criteria is arbitrary and completely depends on what is the most relevant in relation to your use case.

Final result

Now that we have all the pieces working, we will stitch everything together.

We can also add a fallback option to do a product name/title similarity search if we can't find relevant entities in the user prompt.

We will explore 2 options, one with a Langchain agent for a conversational experience, and one that is more deterministic based on code only.

Depending on your use case, you might choose one or the other option and tailor it to your needs.

Building a Langchain agent

We will create a Langchain agent to handle conversations and probing the user for more context.

We need to define exactly how the agent should behave, and give it access to our query and similarity search tools.

Building a code-only experience

As our experiments show, using an agent for this type of task might not be the best option.

Indeed, the agent seems to retrieve results from the tools, but comes up with made-up responses.

For this specific use case, if the conversational aspect is less relevant, we can actually create a function that will call our previously-defined tasks and provide an answer.

Conclusion

User experience

When the primary objective is to extract specific information from our database, Large Language Models (LLMs) can significantly enhance our querying capabilities.

However, it's crucial to base much of this process on robust code logic to ensure a foolproof user experience.

For crafting a genuinely conversational chatbot, further exploration in prompt engineering is necessary, possibly incorporating few-shot examples. This approach helps mitigate the risk of generating inaccurate or misleading information and ensures more precise responses.

Ultimately, the design choice depends on the desired user experience. For instance, if the aim is to create a visual recommendation system, the importance of a conversational interface is less relevant.

Working with a knowledge graph

Retrieving content from a knowledge graph adds complexity but can be useful if you want to leverage connections between items.

The querying part of this notebook would work on a relational database as well, the knowledge graph comes in handy when we want to couple the results with similar items that the graph is surfacing.

Considering the added complexity, make sure using a knowledge graph is the best option for your use case.
If it is the case, feel free to refine what this cookbook presents to match your needs and perform even better!