Generate Embeddings with Azure OpenAI
A notebook showing how to generate embeddings through the Azure OpenAI service, from resource setup to a sample request.
Why it matters
Leverage Azure OpenAI Service to generate embeddings for your data. This asset demonstrates how to set up API connections and create embeddings using Azure.
Outcomes
What it gets done
Configure Azure OpenAI API credentials and endpoints.
Create and list deployments for embedding models.
Generate embeddings for sample text data.
Integrate with Azure OpenAI for embedding generation.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/oai-embeddings | bash Steps
Steps in the chain
Overview
Azure embeddings example
A notebook covering Azure OpenAI embeddings setup - resource endpoint configuration, API key or Active Directory authentication, model deployment, and a sample embedding request. Use it as a conceptual reference for the Azure-specific setup steps (endpoint, deployment, auth) - it targets a pre-1.0 openai library version, so treat it as a walkthrough rather than copy-paste-ready code.
What it does
This notebook walks through generating embeddings via the Azure OpenAI service. Setup starts with installing a pinned pre-1.0 version of the openai library, then configuring api_base (the resource's Endpoint value, found under Resource Management > Keys and Endpoints in the Azure Portal) and api_version. Authentication can go two ways: a key from the Azure Portal (api_type set to azure), or Microsoft Active Directory Authentication (api_type set to azure_ad), which issues a token that expires and needs refreshing via a requests.auth hook. For development it recommends setting these as environment variables instead of hardcoding them in code:
OPENAI_API_BASE
OPENAI_API_KEY
OPENAI_API_TYPE
OPENAI_API_VERSION
Before embeddings can be generated, a model deployment is required - the notebook demonstrates creating one manually in the Azure Portal using the text-similarity-curie-001 model under Resource Management > Model deployments, and separately shows how to list existing deployments in a subscription (useful since creating a new one takes a while). With a deployment in hand, it sends a sample embedding request to it.
When to use - and when NOT to
Use it as a reference for wiring up Azure-hosted OpenAI embeddings specifically - portal-based resource and deployment setup, and both key-based and Active Directory authentication paths. The notebook itself flags that a newer version of the openai library is available and this example targets the older pre-1.0 API surface, so treat it as a conceptual walkthrough of the Azure setup steps (endpoint, deployment, auth) rather than a copy-paste-ready snippet for a current SDK version.
Inputs and outputs
Input is Azure resource credentials (endpoint, API key or Active Directory token, API version) and a deployed embedding model. Output is a vector embedding returned from the deployment for a sample input.
Integrations
It integrates with the Azure OpenAI service via the Azure Portal for resource and deployment management, and with Microsoft Active Directory for token-based authentication as an alternative to API keys.
Who it's for
Developers setting up Azure-hosted OpenAI embeddings for the first time who need to understand the resource, deployment, and authentication steps specific to Azure rather than the standard OpenAI API.
Source README
Azure embeddings example
Note: There is a newer version of the openai library available. See https://github.com/openai/openai-python/discussions/742
This example will cover embeddings using the Azure OpenAI service.
Setup
First, we install the necessary dependencies.
! pip install "openai>=0.28.1,<1.0.0"
For the following sections to work properly we first have to setup some things. Let's start with the api_base and api_version. To find your api_base go to https://portal.azure.com, find your resource and then under "Resource Management" -> "Keys and Endpoints" look for the "Endpoint" value.
We next have to setup the api_type and api_key. We can either get the key from the portal or we can get it through Microsoft Active Directory Authentication. Depending on this the api_type is either azure or azure_ad.
Setup: Portal
Let's first look at getting the key from the portal. Go to https://portal.azure.com, find your resource and then under "Resource Management" -> "Keys and Endpoints" look for one of the "Keys" values.
Note: In this example, we configured the library to use the Azure API by setting the variables in code. For development, consider setting the environment variables instead:
OPENAI_API_BASE
OPENAI_API_KEY
OPENAI_API_TYPE
OPENAI_API_VERSION
(Optional) Setup: Microsoft Active Directory Authentication
Let's now see how we can get a key via Microsoft Active Directory Authentication. Uncomment the following code if you want to use Active Directory Authentication instead of keys from the portal.
A token is valid for a period of time, after which it will expire. To ensure a valid token is sent with every request, you can refresh an expiring token by hooking into requests.auth:
Deployments
In this section we are going to create a deployment that we can use to create embeddings.
Deployments: Create manually
Let's create a deployment using the text-similarity-curie-001 model. Create a new deployment by going to your Resource in your portal under "Resource Management" -> "Model deployments".
Deployments: Listing
Now because creating a new deployment takes a long time, let's look in the subscription for an already finished deployment that succeeded.
Embeddings
Now let's send a sample embedding to the deployment.
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.