What does this prompt chain do?

It tests OpenAI's audio-capable models using promptfoo, covering both speech-to-text (audio input processing) and speech-to-speech (audio output generation) with real sample audio clips from Neil Armstrong's moon-landing speech and JFK's "Ich bin ein Berliner" speech.

What do I need to run this example?

You need an `OPENAI_API_KEY`, and the example includes sample audio files (Armstrong_Small_Step.mp3 and Kennedy_berliner.mp3) in the assets directory, with configuration defined in promptfooconfig.yaml.

How do I set up and run this example?

Run `npx promptfoo@latest init --example openai-audio` to set up, then execute `npx promptfoo eval -c examples/openai-audio/promptfooconfig.yaml` from promptfoo's root directory.

When should I use this example?

Use it when you want to evaluate how an OpenAI audio-capable model transcribes spoken audio input or generates speech output; do not use it if you only need text-based OpenAI evaluation.

Prompt Chain

Generate Code from OpenAI Audio Transcripts

Name: Openai Audio
Availability: OnlineOnly
Author: Promptfoo

A Promptfoo example testing OpenAI's audio-capable models on speech-to-text input and speech-to-speech output.

Copy chain

Works with openai

Promptfoo

Own this? Claim it

Spark score

out of 100

Updated 3 days ago

Version 0.121.19

Models

gpt 4o

Add to Favorites

Why it matters

Leverage OpenAI's audio capabilities to transcribe spoken language and generate corresponding code snippets, streamlining development workflows.

Outcomes

What it gets done

Transcribe audio input using OpenAI's models.

Generate code based on transcribed audio content.

Summarize the transcribed audio for context.

Install

Add it to your toolbox

Run in your project directory:

curl -fsSL https://spark.entire.vc/get/pfoo-openai-audio | bash

Steps

Steps in the chain

Set OpenAI API key

Review sample audio files

Overview

Openai Audio

A Promptfoo example testing OpenAI's audio-capable models on speech-to-text input processing and speech-to-speech output generation. Use it to evaluate OpenAI models on audio tasks - transcription or generated speech - rather than text-only prompts.

What it does

A Promptfoo example testing OpenAI's audio-capable models - both audio input processing (speech-to-text) and audio output generation (speech-to-speech).

When to use - and when NOT to

Use it as a starting point for evaluating OpenAI models on audio tasks - transcribing spoken audio or generating spoken responses - rather than text-only prompts. It requires an OPENAI_API_KEY set as an environment variable before running.

Inputs and outputs

Set up with:

npx promptfoo@latest init --example openai-audio
cd openai-audio
export OPENAI_API_KEY=your-api-key-here
npx promptfoo eval -c examples/openai-audio/promptfooconfig.yaml

Ships two sample audio files under assets/ to test against - Armstrong_Small_Step.mp3 (Neil Armstrong's moon landing speech) and Kennedy_berliner.mp3 (JFK's "Ich bin ein Berliner" speech) - plus promptfooconfig.yaml (providers and tests) and audio-input.json (a JSON template for the audio input prompt).

Integrations

Integrates with OpenAI's audio-capable models via Promptfoo's OpenAI provider, for both speech-to-text and speech-to-speech evaluation.

Who it's for

Teams evaluating OpenAI's audio capabilities - transcription accuracy or generated speech quality - who want a ready-made Promptfoo example with real sample audio rather than building test fixtures from scratch.

Source README

openai-audio (OpenAI Audio Example)

You can run this example with:

npx promptfoo@latest init --example openai-audio
cd openai-audio

This example demonstrates how to use promptfoo to test OpenAI's audio capabilities using audio-capable models. The example focuses on audio input processing (speech-to-text) and audio output generation (speech-to-speech).

Quick Start

You can initialize this example in a new directory using:

npx promptfoo@latest init --example openai-audio

This will create all necessary files and folder structure to get started quickly.

Setup

Set your OpenAI API key as an environment variable:

export OPENAI_API_KEY=your-api-key-here

The example includes sample audio files in the assets directory:
- Armstrong_Small_Step.mp3 - Neil Armstrong's moon landing speech
- Kennedy_berliner.mp3 - JFK's "Ich bin ein Berliner" speech

Files

promptfooconfig.yaml: Configuration file defining the providers and tests
audio-input.json: JSON template for the audio input prompt

Running the Example

From the root directory of promptfoo, run:

npx promptfoo eval -c examples/openai-audio/promptfooconfig.yaml

FAQ

Common questions

Discussion