What are the syntax differences between OpenAI and Anthropic for structured outputs?

OpenAI and Azure OpenAI use `response_format` with `json_schema` at the prompt level (including `type: json_schema`, a `name`, `strict: true`, and a `schema` block), while Anthropic uses `output_format` with `json_schema` at the provider level with the schema nested directly rather than inside an extra `json_schema` wrapper object.

What API keys do I need to run this example?

You need at least one of `OPENAI_API_KEY` or `ANTHROPIC_API_KEY`, settable via `.env` or the shell. If using Azure OpenAI, you'll also need to configure Azure credentials.

What does this prompt chain teach you?

It teaches how to enforce JSON schemas across different providers, the syntax differences between OpenAI's and Anthropic's structured-output configuration, how to validate that responses are always valid JSON objects, and how to require specific fields and types in LLM responses.

When should I use this example?

Use this when you need reliable, schema-validated JSON from an LLM and want to compare how the same schema requirement is configured across multiple providers, or when you need to guarantee specific required fields and types in responses.

Prompt Chain

Generate Structured Code Outputs

Name: Config Structured Outputs
Availability: OnlineOnly
Author: Promptfoo

A promptfoo example enforcing JSON-schema structured outputs across OpenAI, Azure OpenAI, and Anthropic.

Copy chain

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 3 days ago

Version 0.121.19

Models

claudegpt 4o

Add to Favorites

Why it matters

This prompt chain helps developers generate structured outputs from LLMs, enabling more reliable and predictable code generation and data extraction for various applications.

Outcomes

What it gets done

Configure LLMs to produce structured data formats.

Extract specific information from unstructured text.

Generate code snippets based on structured prompts.

Summarize LLM responses into defined schemas.

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-config-structured-outputs | bash

Steps

Steps in the chain

Review and customize promptfooconfig.yaml

Remove unsupported providers

Run the evaluation

View the results

Overview

Config Structured Outputs

This promptfoo example enforces JSON-schema structured outputs across OpenAI, Azure OpenAI (response_format/json_schema) and Anthropic (output_format/json_schema), using two math-problem prompt configs to show step-by-step versus final-answer schemas. Use it when LLM output needs guaranteed valid JSON with required fields and types across multiple providers whose structured-output syntax differs.

What it does

This example demonstrates how to enforce structured JSON outputs using schema validation across multiple AI providers: OpenAI and Azure OpenAI using response_format with json_schema, and Anthropic using output_format with json_schema. It includes two prompt configurations for solving quirky math problems - one requiring step-by-step problem solving with a steps array, and one requiring only the final answer without it.

npx promptfoo@latest init --example config-structured-outputs
cd config-structured-outputs

When to use - and when NOT to

Use this when you need LLM responses that are guaranteed valid JSON with specific required fields and types, and want to compare or support multiple providers' different structured-output syntax in one eval. It requires at least one of OPENAI_API_KEY or ANTHROPIC_API_KEY (or Azure OpenAI credentials per promptfoo's Azure provider docs), set via a .env file or the environment directly.

Inputs and outputs

OpenAI and Azure define the schema at the prompt level:

prompts:
  - raw: 'Your prompt here'
    config:
      response_format:
        type: json_schema
        json_schema:
          name: schema_name
          strict: true
          schema:
            # Your schema here

Anthropic instead defines it at the provider level with output_format (no nested json_schema object, just type: json_schema and schema directly under the provider's config). To get started: review and customize promptfooconfig.yaml, remove any providers you lack API keys for, run promptfoo eval, and view results with promptfoo view.

Who it's for

Developers who need enforced-JSON, schema-validated LLM output and want to test or compare that behavior across OpenAI, Azure OpenAI, and Anthropic in a single promptfoo config, understanding the syntax differences between providers along the way. Running through the example teaches four things specifically: how to enforce JSON schemas across different providers, the syntax differences between OpenAI-style and Anthropic-style structured outputs, how to validate that responses are always valid JSON objects, and how to require specific fields and types in LLM responses.

Source README

config-structured-outputs (Multi-Provider Structured Outputs)

You can run this example with:

npx promptfoo@latest init --example config-structured-outputs
cd config-structured-outputs

This example demonstrates how to enforce structured JSON outputs using schema validation across multiple AI providers:

OpenAI - using response_format with json_schema
Azure OpenAI - using response_format with json_schema
Anthropic - using output_format with json_schema

The example includes two prompt configurations for solving quirky math problems:

One that requires step-by-step problem solving (with steps array)
One that only requires the final answer (without steps array)

Environment Variables

This example requires at least one of the following API keys:

OPENAI_API_KEY - Your OpenAI API key for testing with GPT models
ANTHROPIC_API_KEY - Your Anthropic API key for testing with Claude models
For Azure OpenAI: See Azure OpenAI provider docs for required credentials

You can set these in a .env file or directly in your environment:

export OPENAI_API_KEY=your_api_key_here
export ANTHROPIC_API_KEY=your_api_key_here

Provider Differences

OpenAI/Azure Format

OpenAI and Azure use response_format at the prompt level:

prompts:
  - raw: 'Your prompt here'
    config:
      response_format:
        type: json_schema
        json_schema:
          name: schema_name
          strict: true
          schema:
            # Your schema here

Anthropic Format

Anthropic uses output_format at the provider level:

providers:
  - id: anthropic:messages:claude-sonnet-4-6
    config:
      output_format:
        type: json_schema
        schema:
          # Your schema here (no nested json_schema object)

Getting Started

Review and customize promptfooconfig.yaml as needed
Remove any providers you don't have API keys for
Run the evaluation:
```
promptfoo eval
```
View the results:
```
promptfoo view
```

What You'll Learn

How to enforce JSON schemas across different providers
The syntax differences between OpenAI and Anthropic structured outputs
How to validate that responses are always valid JSON objects
How to require specific fields and types in LLM responses

Learn More

FAQ

Common questions

Discussion