Generate Subtitles from Audio and Transcripts
A promptfoo example aligning audio to transcripts with ElevenLabs Forced Alignment, outputting JSON or SRT subtitles.
Why it matters
Automate the creation of time-aligned subtitles (SRT/VTT) for audio and transcript content. This asset leverages ElevenLabs' forced alignment capabilities to accurately synchronize spoken words with timestamps.
Outcomes
What it gets done
Process audio files to extract speech.
Utilize ElevenLabs for accurate speech-to-text alignment.
Generate SRT and VTT subtitle files based on aligned data.
Ensure precise timing for subtitles matching the audio.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/pfoo-elevenlabs-alignment | bash Overview
Elevenlabs Alignment
This promptfoo example runs ElevenLabs Forced Alignment on audio/transcript pairs, producing either JSON word alignment or SRT subtitle output, validated with format-specific assertions. Use it when you already have a transcript and need time-aligned subtitles or word timing from the matching audio; it does not generate transcripts from audio alone.
What it does
This promptfoo example evaluates ElevenLabs Forced Alignment, which takes an audio file and its transcript and produces time-aligned output - either a JSON structure of aligned words or SRT-format subtitles. Two providers, elevenlabs:alignment:json and elevenlabs:alignment:srt, expose the two output formats, and each test case supplies an audioFile and transcript pair to be aligned.
When to use - and when NOT to
Use this example when you need to generate subtitles or word-level timing data from an existing audio recording and its known transcript - for instance, subtitle generation for video content where the spoken text is already known. It is an alignment tool, not a transcription tool: it requires you to already have the transcript text; if you only have audio and need the transcript generated first, this is not the right building block on its own.
Inputs and outputs
The shared prompt is '{{transcript}}', with audioFile and transcript passed as test variables (a format var also appears in the test data). A defaultTest block asserts not-contains: error on every test. Format-specific tests add their own checks - the JSON provider's output is checked for a words key, and the SRT provider's output is checked for SRT timing markers and matching text:
providers:
# Basic alignment (JSON output)
- id: elevenlabs:alignment:json
label: Alignment (JSON)
# SRT subtitle format
- id: elevenlabs:alignment:srt
label: Alignment (SRT Subtitles)
tests:
- description: Align Armstrong to SRT format
vars:
audioFile: examples/elevenlabs-stt/audio/sample1.mp3
transcript: "That's one small step for man, one giant leap for mankind."
format: srt
assert:
- type: javascript
value: output.includes('-->') && output.includes('small step')
Integrations
Runs against ElevenLabs' alignment providers (elevenlabs:alignment:json and elevenlabs:alignment:srt) through promptfoo, and uses sample audio files from the companion elevenlabs-stt example.
Who it's for
Developers building subtitle-generation or transcript-to-audio-sync pipelines who want to validate ElevenLabs Forced Alignment output in either JSON or SRT format before wiring it into a production video or captioning workflow.
Source README
yaml-language-server: $schema=../../site/static/config-schema.json
description: ElevenLabs Forced Alignment - Subtitle generation
Alignment uses audio files + transcripts - pass via vars
prompts:
- '{{transcript}}'
providers:
Basic alignment (JSON output)
- id: elevenlabs:alignment:json
label: Alignment (JSON)
SRT subtitle format
- id: elevenlabs:alignment:srt
label: Alignment (SRT Subtitles)
Default test configuration
defaultTest:
All tests will require alignment to complete
assert:
- type: not-contains
value: error
tests:
description: Align Armstrong moon landing speech
vars:
audioFile: examples/elevenlabs-stt/audio/sample1.mp3
transcript: "That's one small step for man, one giant leap for mankind."
format: json
assert:- type: javascript
value: output.includes('words') - type: not-contains
value: error
- type: javascript
description: Align Armstrong to SRT format
vars:
audioFile: examples/elevenlabs-stt/audio/sample1.mp3
transcript: "That's one small step for man, one giant leap for mankind."
format: srt
assert:- type: javascript
value: output.includes('-->') && output.includes('small step')
- type: javascript
description: Align sample2 hello message
vars:
audioFile: examples/elevenlabs-stt/audio/sample2.wav
transcript: "Hello. What's today's date? Could you please let me know?"
format: json
assert:- type: javascript
value: output.includes('words') - type: not-contains
value: error
- type: javascript
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.