Why is streaming not recommended for promptfoo evaluations?

Streaming is not recommended because promptfoo waits for the full response before scoring regardless of streaming, so there is no progressive display during evals. Additionally, streaming formats (SSE/chunked) require extra parsing complexity, and end-to-end latency is similar to a non-streaming call.

When should I use this streaming example?

Use this example when you specifically need to validate that a streaming HTTP endpoint works correctly with promptfoo's HTTP provider, or need to parse SSE/chunked output for another reason.

What credentials and configuration does this example require?

This example requires an `OPENAI_API_KEY` (settable in the shell or a `.env` file) and uses `promptfooconfig.yaml` to configure the HTTP provider against the streaming endpoint.

How do I run this streaming example?

Run `npx promptfoo@latest init --example provider-http/streaming`, then navigate to the directory and execute `npx promptfoo@latest eval -c examples/provider-http/streaming/promptfooconfig.yaml` to run the evaluation.

Prompt Chain

Stream OpenAI API responses via HTTP provider

Name: Streaming
Availability: OnlineOnly
Author: Promptfoo

Promptfoo HTTP provider example for OpenAI streaming - and why streaming targets do not help evaluations.

Copy chain

Works with openai

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 26 days ago

Version code-scan-action-0.1

Models

gpt 4o

Add to Favorites

Why it matters

Enable real-time streaming of OpenAI API responses through an HTTP provider interface, allowing developers to implement progressive content delivery in their applications without managing complex WebSocket connections.

Outcomes

What it gets done

Configure HTTP provider to connect with OpenAI's streaming API endpoint

Handle chunked transfer encoding for real-time response delivery

Parse and process server-sent events from the streaming API

Integrate streaming responses into existing HTTP-based workflows

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-streaming | bash

Steps

Steps in the chain

Set your API key

Run the evaluation

View results

Overview

Streaming

A Promptfoo example wiring an HTTP provider to OpenAI's streaming API, documenting why streaming targets add parsing complexity without improving eval speed or display. Use when configuring Promptfoo's HTTP provider for a streaming endpoint or deciding whether streaming is worth it for evals.

What it does

A Promptfoo example demonstrating how to configure an HTTP provider to hit OpenAI's streaming API for evaluation. It's initialized via the standard Promptfoo example scaffolding command and run with the standard eval CLI pointed at the example's config file. The example doubles as an explicit warning against using streaming in evals: Promptfoo's HTTP-provider evaluations wait for the full response before scoring regardless of whether the target streams, so streaming a target buys no progressive display during the eval, adds parsing complexity for SSE or chunked response formats, and produces roughly the same end-to-end latency as a non-streaming call.

When to use - and when NOT to

Use when you need a working reference for wiring Promptfoo's HTTP provider to a streaming OpenAI endpoint, or specifically to understand why streaming targets don't actually help evaluation workflows. Not the pattern to reach for if you're optimizing eval speed or building a progressive-display UI on top of Promptfoo - the example itself documents that streaming provides no benefit in this context, so a non-streaming HTTP provider is the better default for evals.

Inputs and outputs

Input is an OPENAI_API_KEY environment variable, set in the shell or a project-level .env file. Output, after running the eval command against the example's promptfooconfig.yaml, is the standard Promptfoo evaluation result set, viewable via the Promptfoo view command.

npx promptfoo@latest init --example provider-http/streaming
cd provider-http/streaming

Integrations

Built on Promptfoo's HTTP provider talking to OpenAI's streaming API; points to Promptfoo's own HTTP provider documentation for further configuration options.

Who it's for

Developers evaluating an HTTP-based LLM provider with Promptfoo who need a concrete streaming-provider example, or who are deciding whether to stream and want the documented reasoning for why not to.

Source README

provider-http/streaming (HTTP Provider Streaming Example)

This example shows how to use OpenAI's streaming API via HTTP provider.

You can run this example with:

npx promptfoo@latest init --example provider-http/streaming
cd provider-http/streaming

⚠️ Streaming is not recommended for evaluations

Promptfoo supports streaming HTTP targets, but evals wait for full responses before scoring. That means:

No progressive display during evals
Extra parsing complexity for streaming formats (SSE/chunked)
Similar end-to-end latency vs. non-streaming

Environment Variables

Required:

OPENAI_API_KEY - Your OpenAI API key from https://platform.openai.com/api-keys

You can set it in your shell or in a project-level .env file (recommended):

export OPENAI_API_KEY="your-openai-api-key"
### or in .env
OPENAI_API_KEY=your-openai-api-key

Quick Start

Set your API key (or ensure .env is populated)

Run the evaluation (recommended):

npx promptfoo@latest eval -c examples/provider-http/streaming/promptfooconfig.yaml

View results (optional):
```
npx promptfoo@latest view
```

For more HTTP provider configuration options, see the docs: https://promptfoo.dev/docs/providers/http.

FAQ

Common questions

Discussion