What OpenAI image models does this prompt chain evaluate?

This prompt chain evaluates four OpenAI image-generation models: GPT Image 2 (recommended with flexible custom sizes), GPT Image 1.5 (strong instruction following with token-based pricing), GPT Image 1, and GPT Image 1 Mini (cost-efficient). It tests them through promptfoo's `openai:image` provider for text-to-image generation.

What are the pricing differences between OpenAI's image models at 1024x1024?

Pricing ranges from $0.005 per image for GPT Image 1 Mini at low quality to $0.211 for GPT Image 2 at high quality. GPT Image 1.5 costs approximately $0.064–$0.192 depending on quality, while GPT Image 1 ranges from $0.011–$0.167.

What image sizes and formats does this support?

Supported sizes include 1024x1024, 1024x1536, 1536x1024, and auto (with GPT Image 2 also supporting custom WIDTHxHEIGHT). Output formats include PNG, JPEG, and WebP, with configurable compression for JPEG and WebP.

What are the limitations of this prompt chain?

This example only supports text-to-image generation and does not support image edits, reference-input variations, or streaming partial images. These other modes are rejected if configured.

How do I set up and run this prompt chain?

Set up via `npx promptfoo@latest init --example openai-images`, then run `promptfoo eval` and `promptfoo view`. You'll need to provide your `OPENAI_API_KEY`, and each model is configured with its own provider ID for side-by-side comparison.

Prompt Chain

Generate Images with OpenAI

Name: Openai Images
Availability: OnlineOnly
Author: Promptfoo

A promptfoo example comparing OpenAI's GPT Image models on text-to-image generation, pricing, and quality.

Copy chain

Works with openai

Promptfoo

Maintainer?

Spark score

out of 100

Updated 13 days ago

Version 0.121.18

Models

gpt 4o

Add to Favorites

Why it matters

Leverage OpenAI's image generation capabilities to create visual content. This prompt chain automates the process of sending requests and receiving image outputs.

Outcomes

What it gets done

Configure image generation parameters

Send prompts to OpenAI API

Receive and process generated images

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-openai-images | bash

Steps

Steps in the chain

Create the example

Set your API key

Run the evaluation

View the results

Overview

Openai Images

A promptfoo example comparing OpenAI's GPT Image 2, 1.5, 1, and 1 Mini text-to-image models on size, quality, output format, and per-image pricing. Use it to compare GPT Image model quality and cost trade-offs. Supports text-to-image only, not edits, variations, or streaming. DALL-E 2/3 remain supported but are deprecated.

What it does

This promptfoo example evaluates OpenAI's current image generation models - GPT Image 2, GPT Image 1.5, GPT Image 1, and GPT Image 1 Mini - side by side, comparing artistic style prompts across models with different image sizes and quality settings. It uses text-to-image generation through the Image API's generations endpoint only; image edits, reference inputs, variations, and streaming partial images are rejected if configured, since they aren't part of what this example or promptfoo's openai:image provider currently supports.

When to use - and when NOT to

Use this example when you want to compare OpenAI's newer GPT Image models on quality, size, and cost trade-offs for a given prompt style. GPT Image 2 is the recommended latest model, with flexible custom sizes and improved output controls; GPT Image 1.5 offers strong instruction following and photorealistic quality on token-based pricing; GPT Image 1 Mini is a cost-efficient version of GPT Image 1 with the same capabilities at lower cost. dall-e-3 and dall-e-2 remain supported for backward compatibility but are deprecated by OpenAI - use one of the GPT Image models for any new eval instead.

Inputs and outputs

Each model is configured with size, quality, background, output_format, output_compression, moderation, and (for GPT Image 2) n and user:

providers:
  - id: openai:image:gpt-image-2
    config:
      size: 1024x1024 # common sizes, custom WIDTHxHEIGHT, auto
      quality: low # low, medium, high, auto
      background: opaque # opaque, auto
      output_format: webp # png, jpeg, webp
      output_compression: 80 # 0-100, only set with jpeg/webp

Per-image pricing at 1024x1024 varies sharply by model and quality: GPT Image 2 runs $0.006 (low) to $0.211 (high), GPT Image 1.5 roughly $0.064 to $0.192, GPT Image 1 $0.011 to $0.167, and GPT Image 1 Mini $0.005 to $0.036. For GPT Image 2's auto quality or custom sizes, promptfoo preserves OpenAI's usage metadata and leaves cost unset rather than estimating it.

Integrations

Set OPENAI_API_KEY, then run promptfoo eval followed by promptfoo view to compare model outputs side by side. One configuration difference worth checking before switching models: GPT Image 2's background option only accepts opaque or auto, while GPT Image 1.5, GPT Image 1, and GPT Image 1 Mini also support transparent - a config that relies on transparent backgrounds won't carry over to GPT Image 2 unchanged. Full provider configuration is documented in OpenAI's Image Generation API guide and promptfoo's OpenAI provider docs.

Who it's for

Teams choosing among OpenAI's image generation models who need a working cost-and-quality comparison across GPT Image 2, 1.5, 1, and 1 Mini before committing to one for production. Prices listed are 1024x1024 output estimates only and exclude input text token costs, so actual per-eval spend will run somewhat higher once prompt tokens are counted in.

Source README

openai-images (OpenAI Image Generation Example)

You can run this example with:

npx promptfoo@latest init --example openai-images
cd openai-images

A simple example showing how to evaluate OpenAI's current image generation models (GPT Image 2, GPT Image 1.5, GPT Image 1, GPT Image 1 Mini) with promptfoo.

Quick Start

### Create this example
npx promptfoo@latest init --example openai-images

### Set your API key
export OPENAI_API_KEY=your-key-here

### Run the evaluation
promptfoo eval

### View the results
promptfoo view

What's in this Example

Compares GPT Image 2, GPT Image 1.5, GPT Image 1, and GPT Image 1 Mini outputs
Uses text-to-image generation through the Image API generations endpoint
Tests artistic style prompts across different models
Configures different image sizes and quality settings
Tests with different subjects

Supported Models

GPT Image 2 (Recommended)

OpenAI's latest image generation model with flexible custom sizes and improved output controls.

providers:
  - id: openai:image:gpt-image-2
    config:
      size: 1024x1024 # common sizes, custom WIDTHxHEIGHT, auto
      quality: low # low, medium, high, auto
      background: opaque # opaque, auto
      output_format: webp # png, jpeg, webp
      output_compression: 80 # 0-100, only set with jpeg/webp
      moderation: auto # auto, low
      n: 1 # 1-10 images
      user: promptfoo-user # optional end-user identifier

GPT Image 1.5

High-quality GPT Image model with strong instruction following, prompt adherence, and photorealistic quality. Uses token-based pricing.

providers:
  - id: openai:image:gpt-image-1.5
    config:
      size: 1024x1024 # 1024x1024, 1024x1536, 1536x1024, auto
      quality: low # low, medium, high, auto
      background: transparent # transparent, opaque, auto
      output_format: webp # png, jpeg, webp
      output_compression: 80 # 0-100, only set with jpeg/webp
      moderation: auto # auto, low

GPT Image 1

High-quality image generation model with superior instruction following and text rendering.

providers:
  - id: openai:image:gpt-image-1
    config:
      size: 1024x1024 # 1024x1024, 1024x1536, 1536x1024, auto
      quality: low # low, medium, high, auto
      background: transparent # transparent, opaque, auto
      output_format: webp # png, jpeg, webp
      output_compression: 80 # 0-100, only set with jpeg/webp
      moderation: auto # auto, low

GPT Image 1 Mini

Cost-efficient version of GPT Image 1 with the same capabilities at lower cost.

providers:
  - id: openai:image:gpt-image-1-mini
    config:
      size: 1024x1024 # 1024x1024, 1024x1536, 1536x1024, auto
      quality: low # low, medium, high, auto
      background: transparent # transparent, opaque, auto
      output_format: webp # png, jpeg, webp
      output_compression: 80 # 0-100, only set with jpeg/webp
      moderation: auto # auto, low

DALL-E 3 and DALL-E 2

dall-e-3 and dall-e-2 remain supported by the provider for backward compatibility, but they are deprecated by OpenAI. Use gpt-image-2, gpt-image-1.5, gpt-image-1, or gpt-image-1-mini for new evals.

Promptfoo's openai:image provider currently supports text-to-image generation. Image edits/reference inputs, variations, and streaming partial images are not part of this example and are rejected if configured.

Pricing

Model	Quality	Size	Price per image
GPT Image 2	Low	1024x1024	$0.006
GPT Image 2	Medium	1024x1024	$0.053
GPT Image 2	High	1024x1024	$0.211
GPT Image 1.5	Low	1024x1024	~$0.064
GPT Image 1.5	Medium	1024x1024	~$0.128
GPT Image 1.5	High	1024x1024	~$0.192
GPT Image 1	Low	1024x1024	$0.011
GPT Image 1	Medium	1024x1024	$0.042
GPT Image 1	High	1024x1024	$0.167
GPT Image 1 Mini	Low	1024x1024	$0.005
GPT Image 1 Mini	Medium	1024x1024	$0.011
GPT Image 1 Mini	High	1024x1024	$0.036

Note: Prices shown are 1024x1024 output image estimates and do not include input text tokens. For GPT Image 2 auto quality or custom sizes, promptfoo preserves OpenAI usage metadata and leaves cost unset instead of guessing.