Evaluate DeepSeek models on Azure AI Foundry

Azure DeepSeek is a promptfoo example demonstrating how to evaluate DeepSeek models deployed on Azure AI Foundry, including the DeepSeek-R1 reasoning model. It covers four available models: DeepSeek-R1 (an advanced reasoning model), DeepSeek-V3 (a standard chat model), and two distilled reasoning variants, DeepSeek-R1-Distill-Llama-70B and DeepSeek-R1-Distill-Qwen-32B.

Use it when running promptfoo evaluations against DeepSeek models hosted on Azure AI Foundry rather than DeepSeek's own API or another cloud provider. Reasoning models like DeepSeek-R1 need distinct configuration from standard chat models - the example documents this explicitly:

providers:
  - id: azure:chat:DeepSeek-R1
    config:
      isReasoningModel: true # Required for reasoning models
      max_completion_tokens: 4096 # Use instead of max_tokens
      reasoning_effort: medium # low, medium, or high

It is not applicable to non-Azure DeepSeek deployments, which would use a different promptfoo provider configuration.

Input is a DeepSeek model already deployed in Azure AI Foundry, accessed via AZURE_API_KEY and AZURE_API_HOST environment variables. The example is scaffolded with npx promptfoo@latest init --example azure/deepseek, and evaluation output is produced by npx promptfoo@latest eval followed by npx promptfoo@latest view to inspect results in the promptfoo UI. For reasoning models, reasoning_effort (low, medium, or high) controls how much reasoning budget DeepSeek-R1 uses per response, and max_completion_tokens replaces the standard max_tokens setting.

Runs through promptfoo's Azure provider, which requires the DeepSeek model to already be deployed in Azure AI Foundry before evaluation. Further reference is available in promptfoo's Azure provider documentation and Microsoft's DeepSeek-on-Azure product page.

Teams evaluating DeepSeek models - standard chat or R1-class reasoning models - deployed through Azure AI Foundry, who need the correct promptfoo provider configuration for reasoning-model-specific parameters.