Question Answering with Langchain, Qdrant and OpenAI

Question Answering with Langchain, Qdrant and OpenAI

This notebook presents how to implement a Question Answering system with Langchain, Qdrant as a knowledge based and OpenAI embeddings. If you are not familiar with Qdrant, it's better to check out the Getting_started_with_Qdrant_and_OpenAI.ipynb notebook.

This notebook presents an end-to-end process of:

1. Calculating the embeddings with OpenAI API.
2. Storing the embeddings in a local instance of Qdrant to build a knowledge base.
3. Converting raw text query to an embedding with OpenAI API.
4. Using Qdrant to perform the nearest neighbour search in the created collection to find some context.
5. Asking LLM to find the answer in a given context.

All the steps will be simplified to calling some corresponding Langchain methods.

Prerequisites

For the purposes of this exercise we need to prepare a couple of things:

1. Qdrant server instance. In our case a local Docker container.
2. The qdrant-client library to interact with the vector database.
3. Langchain as a framework.
4. An OpenAI API key.

Start Qdrant server

We're going to use a local Qdrant instance running in a Docker container. The easiest way to launch it is to use the attached [docker-compose.yaml] file and run the following command:

We might validate if the server was launched successfully by running a simple curl command:

Install requirements

This notebook obviously requires the openai, langchain and qdrant-client packages.

Prepare your OpenAI API key

The OpenAI API key is used for vectorization of the documents and queries.

If you don't have an OpenAI API key, you can get one from https://beta.openai.com/account/api-keys.

Once you get your key, please add it to your environment variables as OPENAI_API_KEY by running following command:

Load data

In this section we are going to load the data containing some natural questions and answers to them. All the data will be used to create a Langchain application with Qdrant being the knowledge base.

Chain definition

Langchain is already integrated with Qdrant and performs all the indexing for given list of documents. In our case we are going to store the set of answers we have.

At this stage all the possible answers are already stored in Qdrant, so we can define the whole QA chain.

Search data

Once the data is put into Qdrant we can start asking some questions. A question will be automatically vectorized by OpenAI model, and the created vector will be used to find some possibly matching answers in Qdrant. Once retrieved, the most similar answers will be incorporated into the prompt sent to OpenAI Large Language Model. The communication between all the services is shown on a graph:

Custom prompt templates

The stuff chain type in Langchain uses a specific prompt with question and context documents incorporated. This is what the default prompt looks like:

Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.
{context}
Question: {question}
Helpful Answer:

We can, however, provide our prompt template and change the behaviour of the OpenAI LLM, while still using the stuff chain type. It is important to keep {context} and {question} as placeholders.

Experimenting with custom prompts

We can try using a different prompt template, so the model:

1. Responds with a single-sentence answer if it knows it.
2. Suggests a random song title if it doesn't know the answer to our question.