Isolate Speech from Audio Noise
Promptfoo example removing background noise from audio with ElevenLabs isolation, chainable into transcription or alignment.
Why it matters
Clean up audio recordings by removing background noise to ensure clear and understandable speech. This asset leverages ElevenLabs' audio isolation capabilities to extract pristine vocal tracks from noisy files.
Outcomes
What it gets done
Remove background noise from audio files
Extract clean speech using ElevenLabs
Process audio for enhanced clarity
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/pfoo-isolation | bash Steps
Steps in the chain
Overview
Isolation
A promptfoo example removing background noise from audio using ElevenLabs audio isolation, with cost tracking and direct chaining into transcription or subtitle alignment. Use when cleaning noisy recordings before transcription, subtitling, or delivery, and you need isolation quality validated through promptfoo assertions.
What it does
This example removes background noise from audio files to extract clean speech using ElevenLabs audio isolation. It takes a noisy audio file and returns a cleaned version with background noise removed, speech preserved and enhanced, consistent quality, and a typically smaller file size. It tests noise removal, audio quality comparison between original and isolated audio, multiple MP3 output bitrates (128kbps and 192kbps), and per-file cost tracking. Typical results show a 15-25 dB signal-to-noise-ratio improvement, 10-30% smaller file size, enhanced speech clarity, and preserved natural voice tones.
When to use - and when NOT to
Use this example for podcast cleanup, interview/call enhancement, speech-to-text preprocessing, voiceover repair, or call-center QA - anywhere noisy recordings need cleaning before further use. It works best with constant background noise (AC hum, fan noise) and preserves multiple speakers' voices while removing only noise; for music mixed with speech, manual editing is recommended first instead. It is not a transcription or subtitle tool itself - those are separate, chainable ElevenLabs examples.
Inputs and outputs
Set ELEVENLABS_API_KEY, then scaffold and run:
npx promptfoo@latest init --example provider-elevenlabs/isolation
cd provider-elevenlabs/isolation
export ELEVENLABS_API_KEY=your_api_key_here
npx promptfoo@latest eval
Supported input formats: MP3, WAV, FLAC, OGG, M4A, OPUS, WebM. Output formats: mp3_44100_128 (128kbps), mp3_44100_192 (192kbps), or pcm_44100 (uncompressed). Config selects a provider like elevenlabs:isolation:basic or elevenlabs:isolation:hq with an outputFormat and an audioFile var. The response includes the base64-encoded isolated audio, original and isolated file sizes, output format, processing latency, and estimated cost; assertions can check for 'isolated successfully', absence of error, and a cost threshold (e.g. max $1.00 per file).
Integrations
Integrates the ElevenLabs Audio Isolation API as promptfoo provider elevenlabs:isolation (with :basic/:hq variants), priced at roughly $0.10 per minute of audio with costs auto-tracked in eval results (free tier varies by plan). It chains directly into other ElevenLabs providers: isolation output can feed elevenlabs:stt for transcription of the cleaned audio, or elevenlabs:alignment to generate subtitles from the cleaned audio via {{previousOutput.audio}}. Unsupported formats can be converted first with ffmpeg -i input.video -vn -acodec mp3 output.mp3, and large files that time out can raise the config's timeout (e.g. to 180000ms / 3 minutes). If isolation reports success but the output sounds unchanged, the likely cause is source audio that was already very clean or had noise too faint to detect - isolation is documented to work best on genuinely noisy recordings, not already-clean ones.
Who it's for
Teams cleaning up noisy recordings - podcasts, interviews, call-center audio, voiceovers - who want isolation quality validated through promptfoo assertions and, where needed, chained directly into transcription or subtitle-alignment pipelines rather than run as a standalone manual step.
Source README
provider-elevenlabs/isolation (ElevenLabs Audio Isolation)
Remove background noise from audio files to extract clean speech using ElevenLabs audio isolation.
Quick Start
npx promptfoo@latest init --example provider-elevenlabs/isolation
cd provider-elevenlabs/isolation
export ELEVENLABS_API_KEY=your_api_key_here
npx promptfoo@latest eval
What this tests
- Noise removal: Extract clean speech from noisy audio
- Audio quality: Compare original vs isolated audio size/quality
- Output formats: MP3 at different bitrates (128kbps, 192kbps)
- Cost tracking: Monitor per-file processing costs
How it works
Audio isolation takes a noisy audio file and returns a cleaned version with:
- Background noise removed
- Speech preserved and enhanced
- Consistent audio quality
- Reduced file size (noise-free)
Use Cases
- Podcast cleanup: Remove background noise from recordings
- Interview enhancement: Clean up phone/video call audio
- STT preprocessing: Improve transcription accuracy
- Voiceover repair: Fix audio recorded in noisy environments
- Call center QA: Enhance customer service call recordings
Supported Formats
Input formats: MP3, WAV, FLAC, OGG, M4A, OPUS, WebM
Output formats:
mp3_44100_128- Standard quality (128kbps)mp3_44100_192- High quality (192kbps)pcm_44100- Uncompressed PCM
Configuration
Basic isolation (MP3)
providers:
- id: elevenlabs:isolation:basic
label: Audio Isolation (MP3)
config:
outputFormat: mp3_44100_128
tests:
- vars:
audioFile: path/to/noisy-audio.mp3
High quality output
providers:
- id: elevenlabs:isolation:hq
label: Audio Isolation (HQ)
config:
outputFormat: mp3_44100_192
Testing Assertions
tests:
- description: Verify isolation succeeds
vars:
audioFile: examples/provider-elevenlabs/stt/audio/sample1.mp3
assert:
- type: javascript
value: output.includes('isolated successfully')
- type: not-contains
value: error
- type: cost
threshold: 1.00 # Max $1.00 per file
What to look for in results
The response includes:
- Isolated audio: Base64-encoded cleaned audio file
- Original size: Size of input audio file
- Isolated size: Size of cleaned audio (typically smaller)
- Format: Output audio format (mp3, pcm, etc.)
- Latency: Processing time in milliseconds
- Cost: Estimated processing cost
Best Practices
- Source quality: Use highest quality source audio available
- Noise type: Works best with constant background noise (AC, fan, hum)
- Multiple speakers: Preserves all speech, removes only noise
- Pre-processing: For music removal, consider manual editing first
Audio Quality Comparison
Isolation typically provides:
- Signal-to-Noise Ratio (SNR): 15-25 dB improvement
- File size reduction: 10-30% smaller (noise-free)
- Speech clarity: Enhanced intelligibility
- Frequency response: Preserved natural voice tones
Cost Information
Audio isolation pricing is based on audio duration:
- ~$0.10 per minute of audio
- Free tier: Varies by plan
The provider automatically tracks costs in evaluation results.
Common Issues
"Audio isolated successfully" but quality unchanged
Possible causes:
- Source audio already very clean
- Noise level too low to detect
- Music/speech mixed with background (try manual editing)
Solution: Check original audio quality - isolation works best with noisy recordings.
Large file processing timeout
Solution: Increase timeout in config:
config:
timeout: 180000 # 3 minutes
Unsupported audio format
Solution: Convert to supported format using ffmpeg:
ffmpeg -i input.video -vn -acodec mp3 output.mp3
Pipeline Integration
Isolation → Transcription
### Step 1: Isolate noisy audio
providers:
- id: elevenlabs:isolation
### Step 2: Transcribe cleaned audio
providers:
- id: elevenlabs:stt
config:
audioFile: '{{previousOutput.audio}}'
Isolation → Alignment
### Step 1: Clean audio
providers:
- id: elevenlabs:isolation
### Step 2: Generate subtitles from clean audio
providers:
- id: elevenlabs:alignment
config:
audioFile: '{{previousOutput.audio}}'
transcript: 'Your transcript here'
Related Examples
- ElevenLabs STT - Transcribe audio to text
- ElevenLabs Alignment - Generate subtitles
Resources
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.