Chat with Your Data on Azure
Ground Azure OpenAI chat models in your own data via Azure Cognitive Search, with no fine-tuning required.
Why it matters
Leverage Azure OpenAI models to chat and analyze your own data without fine-tuning. Ground responses in your specific documents for enhanced accuracy and relevance.
Outcomes
What it gets done
Connect Azure OpenAI chat models to your custom data sources.
Utilize Azure Cognitive Search for efficient data retrieval.
Generate contextually relevant answers based on your documents.
Integrate with Azure Blob Storage for data management.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/oai-chatwithyourowndata | bash Steps
Steps in the chain
Overview
Azure chat completion models with your own data (preview)
An archived Azure OpenAI cookbook showing how to ground chat models in your own documents via Azure Cognitive Search, without fine-tuning, using a managed extensions endpoint. Use it for Microsoft-managed retrieval grounding over a vector-database-based approach. Note: this example is archived and superseded by a newer version of the openai library.
What it does
This archived example (superseded by a newer version of the openai Python library) shows how to use the then-preview Azure OpenAI on your data feature to run chat models like GPT-3.5-Turbo and GPT-4 grounded in your own documents, without training or fine-tuning. Azure Cognitive Search retrieves relevant data from your designated sources based on the user's input and conversation history, augments the prompt with that retrieved content, and resubmits it to the model - so answers draw on the latest information in your data rather than only the model's pretrained knowledge, which also helps avoid outdated or incorrect responses.
When to use - and when NOT to
Use this when you want a pre-built, Microsoft-managed retrieval solution grounding Azure OpenAI chat models in your own documents, rather than building custom retrieval with a vector database yourself (the cookbook points to its vector database examples for that alternative path). It requires an Azure OpenAI resource with a chat model deployed, an Azure Cognitive Search resource, an Azure Blob Storage resource holding your documents, and an index built over those documents via Azure AI Studio's quickstart flow.
Inputs and outputs
Connection details go in a .env file: OPENAI_API_BASE and OPENAI_API_KEY for the Azure OpenAI resource, plus SEARCH_ENDPOINT, SEARCH_KEY, and SEARCH_INDEX_NAME for Cognitive Search. Authentication supports either a plain Azure API key or Microsoft Active Directory tokens, which expire and need refreshing via a hook into requests.auth. The worked example grounds the model in Azure AI services and machine learning documentation uploaded to the search index beforehand.
Integrations
The Python SDK is configured with a convenience function that sets a custom adapter targeting the chat completions extensions endpoint for a given deployment ID, then wires in the search endpoint, key, and index name via a dataSources argument so every question gets grounded in the connected data automatically. Responses include an additional context property showing which data the model actually referenced, and streaming is available via stream=True.
Who it's for
Azure OpenAI users who want retrieval-augmented chat over their own documents through Microsoft's managed Cognitive Search integration rather than building and maintaining a custom vector-database retrieval pipeline themselves. Full guidance on uploading documents to Blob Storage and building the search index through Azure AI Studio is covered in Microsoft's own quickstart, which this example assumes has already been completed before the chat code itself is configured.
Source README
Azure chat completion models with your own data (preview)
Note: There is a newer version of the openai library available. See https://github.com/openai/openai-python/discussions/742
This example shows how to use Azure OpenAI service models with your own data. The feature is currently in preview.
Azure OpenAI on your data enables you to run supported chat models such as GPT-3.5-Turbo and GPT-4 on your data without needing to train or fine-tune models. Running models on your data enables you to chat on top of, and analyze your data with greater accuracy and speed. One of the key benefits of Azure OpenAI on your data is its ability to tailor the content of conversational AI. Because the model has access to, and can reference specific sources to support its responses, answers are not only based on its pretrained knowledge but also on the latest information available in the designated data source. This grounding data also helps the model avoid generating responses based on outdated or incorrect information.
Azure OpenAI on your own data with Azure Cognitive Search provides a customizable, pre-built solution for knowledge retrieval, from which a conversational AI application can be built. To see alternative methods for knowledge retrieval and semantic search, check out the cookbook examples for vector databases.
How it works
Azure OpenAI on your own data connects the model with your data, giving it the ability to retrieve and utilize data in a way that enhances the model's output. Together with Azure Cognitive Search, data is retrieved from designated data sources based on the user input and provided conversation history. The data is then augmented and resubmitted as a prompt to the model, giving the model contextual information it can use to generate a response.
See the Data, privacy, and security for Azure OpenAI Service for more information.
Prerequisites
To get started, we'll cover a few prequisites.
To properly access the Azure OpenAI Service, we need to create the proper resources at the Azure Portal (you can check a detailed guide on how to do this in the Microsoft Docs)
To use your own data with Azure OpenAI models, you will need:
- Azure OpenAI access and a resource with a chat model deployed (for example, GPT-3 or GPT-4)
- Azure Cognitive Search resource
- Azure Blob Storage resource
- Your documents to be used as data (See data source options)
For a full walk-through on how to upload your documents to blob storage and create an index using the Azure AI Studio, see this Quickstart.
Setup
First, we install the necessary dependencies.
In this example, we'll use dotenv to load our environment variables. To connect with Azure OpenAI and the Search index, the following variables should be added to a .env file in KEY=VALUE format:
OPENAI_API_BASE- the Azure OpenAI endpoint. This can be found under "Keys and Endpoints" for your Azure OpenAI resource in the Azure Portal.OPENAI_API_KEY- the Azure OpenAI API key. This can be found under "Keys and Endpoints" for your Azure OpenAI resource in the Azure Portal. Omit if using Azure Active Directory authentication (see belowAuthentication using Microsoft Active Directory)SEARCH_ENDPOINT- the Cognitive Search endpoint. This URL be found on the "Overview" of your Search resource on the Azure Portal.SEARCH_KEY- the Cognitive Search API key. Found under "Keys" for your Search resource in the Azure Portal.SEARCH_INDEX_NAME- the name of the index you created with your own data.
Authentication
The Azure OpenAI service supports multiple authentication mechanisms that include API keys and Azure credentials.
Authentication using API key
To set up the OpenAI SDK to use an Azure API Key, we need to set up the api_type to azure and set api_key to a key associated with your endpoint (you can find this key in "Keys and Endpoints" under "Resource Management" in the Azure Portal)
Authentication using Microsoft Active Directory
Let's now see how we can get a key via Microsoft Active Directory Authentication. See the documentation for more information on how to set this up.
A token is valid for a period of time, after which it will expire. To ensure a valid token is sent with every request, you can refresh an expiring token by hooking into requests.auth:
Chat completion model with your own data
Setting the context
In this example, we want our model to base its responses on Azure AI services documentation data. Following the Quickstart shared previously, we have added the markdown file for the Azure AI services and machine learning documentation page to our search index. The model is now ready to answer questions about Azure AI services and machine learning.
Code
To chat with Azure OpenAI models using your own data with the Python SDK, we must first set up the code to target the chat completions extensions endpoint which is designed to work with your own data. To do this, we've created a convenience function that can be called to set a custom adapter for the library which will target the extensions endpoint for a given deployment ID.
Now we can call the convenience function to configure the SDK with the model we plan to use for our own data.
Providing our search endpoint, key, and index name for the dataSources keyword argument, any questions posed to the model will now be grounded in our own data. An additional property, context, will be provided to show the data the model referenced to answer the question.
If you would prefer to stream the response from the model, you can pass the stream=True keyword argument:
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.