What Llama models does this prompt chain compare?

This prompt chain compares six Meta Llama models deployed on Azure AI Foundry: Llama 4 Maverick (128 experts, FP8), Llama 4 Scout (16 experts), Llama 3.3 70B, and three Llama 3.1 variants (405B, 70B, and 8B). It specifically focuses on comparing Llama 4 Maverick against Llama 4 Scout for code generation tasks.

What environment variables do I need to set up to use this?

You need to set `AZURE_API_KEY` and `AZURE_API_HOST` environment variables pointing to your Llama deployment in Azure AI Foundry.

How do I run the evaluation and view results?

Run `npx promptfoo@latest eval` to send code-generation test cases to both models, then use `npx promptfoo@latest view` to see the results including quality and performance differences.

When should I use this prompt chain?

Use it when you have Llama models already deployed in Azure AI Foundry and want to evaluate trade-offs between model capacity, inference speed, and output quality—particularly when deciding between Llama 4 Maverick and Scout for code generation. Do not use it if you haven't yet deployed Llama models in Azure AI Foundry.

Prompt Chain

Integrate Llama with Azure AI Foundry

Name: Llama
Availability: OnlineOnly
Author: Promptfoo

Promptfoo example comparing Llama 4 Maverick and Scout on Azure AI Foundry for code generation quality and speed.

Copy chain

Works with azure github

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 23 days ago

Version 0.121.19

Models

llama 3

Add to Favorites

Why it matters

Leverage Meta's Llama models on Azure AI Foundry for advanced AI applications. This asset facilitates seamless integration, enabling powerful natural language processing and generation capabilities within your Azure environment.

Outcomes

What it gets done

Connect Llama models to Azure AI Foundry.

Utilize promptfoo for efficient prompt management and testing.

Enable advanced AI functionalities like summarization and chatbots.

Streamline AI development within the Azure ecosystem.

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-llama | bash

Steps

Steps in the chain

Deploy Llama models in Azure AI Foundry

Set environment variables

Run evaluation

View results

Overview

Llama

A promptfoo example for evaluating Meta Llama models on Azure AI Foundry, comparing Llama 4 Maverick and Scout on code generation. Use when evaluating Llama models hosted specifically on Azure AI Foundry with promptfoo; not for non-Azure Llama deployments.

What it does

Azure Llama is a promptfoo example demonstrating how to evaluate Meta Llama models deployed on Azure AI Foundry. It's configured to compare Llama 4 Maverick and Llama 4 Scout specifically on code generation tasks, helping evaluate the trade-off between model capacity (mixture-of-experts count), inference speed, and output quality.

When to use - and when NOT to

Use it when running promptfoo evaluations against Llama models hosted on Azure AI Foundry, particularly when deciding between Llama 4's two expert-count variants for a code-generation workload. Six models are covered: Llama-4-Maverick-17B-128E-Instruct-FP8 (128 experts, FP8), Llama-4-Scout-17B-16E-Instruct (16 experts), Llama-3.3-70B-Instruct, Meta-Llama-3.1-405B-Instruct, Meta-Llama-3.1-70B-Instruct, and Meta-Llama-3.1-8B-Instruct. It is not applicable to Llama deployments outside Azure AI Foundry, which would need a different promptfoo provider configuration.

Inputs and outputs

Input is a Llama model already deployed in Azure AI Foundry, accessed via AZURE_API_KEY and AZURE_API_HOST environment variables. The example is scaffolded with:

npx promptfoo@latest init --example azure/llama
cd azure/llama

Evaluation is run with npx promptfoo@latest eval and results are inspected with npx promptfoo@latest view in the promptfoo UI.

Integrations

Runs through promptfoo's Azure provider, which requires the target Llama model to already be deployed in Azure AI Foundry. Further reference is available in promptfoo's Azure provider documentation and Microsoft's Llama-on-Azure product page.

Who it's for

Teams evaluating Meta Llama models deployed through Azure AI Foundry, especially those deciding between Llama 4 Maverick's higher expert count and Llama 4 Scout's leaner configuration for code-generation quality versus speed.

Source README

azure/llama (Azure Llama Models)

This example demonstrates how to use Meta Llama models on Azure AI Foundry with promptfoo.

You can run this example with:

npx promptfoo@latest init --example azure/llama
cd azure/llama

Setup

Deploy Llama models in Azure AI Foundry
Set your environment variables:

export AZURE_API_KEY=your-api-key
export AZURE_API_HOST=your-deployment.services.ai.azure.com

Available Llama Models

Model	Description
`Llama-4-Maverick-17B-128E-Instruct-FP8`	Llama 4 Maverick (128 experts, FP8)
`Llama-4-Scout-17B-16E-Instruct`	Llama 4 Scout (16 experts)
`Llama-3.3-70B-Instruct`	Llama 3.3 70B
`Meta-Llama-3.1-405B-Instruct`	Llama 3.1 405B
`Meta-Llama-3.1-70B-Instruct`	Llama 3.1 70B
`Meta-Llama-3.1-8B-Instruct`	Llama 3.1 8B

Running the Example

npx promptfoo@latest eval
npx promptfoo@latest view

Configuration

The example compares Llama 4 Maverick and Llama 4 Scout on code generation tasks. This helps evaluate the trade-off between model capacity (expert count), speed, and quality.