Integrate Azure Speech Services for Text and Audio
Transcribe audio to text and synthesize speech via Azure Speech in a LlamaIndex agent.
Why it matters
Leverage Microsoft Azure's advanced speech services to enable agents to transcribe audio files into text and generate audio files from text, streamlining content creation and data processing.
Outcomes
What it gets done
Transcribe audio files (.wav) into text using Azure Speech-to-Text.
Synthesize audio from input text using Azure Text-to-Speech.
Integrate speech capabilities into agent workflows for automated tasks.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/li-tool-tools-azure-speech | bash Overview
Azure Speech Tool
A LlamaIndex tool for Azure Speech services that synthesizes speech from text and transcribes .wav audio into text. Use when an agent needs to speak responses aloud or transcribe and summarize spoken audio content.
What it does
The Azure Speech Tool lets a LlamaIndex agent use Microsoft Azure speech services to transcribe audio files to text and create audio files from text. AzureSpeechToolSpec is initialized with a speech_key and a region (such as eastus), and exposes two functions to the agent.
text_to_speech takes an input string and synthesizes audio that plays on the user's computer. speech_to_text takes a .wav file and transcribes it into text. The source's own worked example shows both directions in one conversation: first asking the agent to say "hello world" aloud, then asking it to summarize the contents of a .wav audio file into a few sentences - showing the tool handling both speech generation and speech understanding within the same agent session.
When to use - and when NOT to
Use it when a LlamaIndex agent needs to speak responses aloud or needs to understand and summarize spoken audio content from a .wav file. Do not use it without an Azure account and a configured speech resource; both speech_key and region are required and the tool has no functionality without valid Azure credentials.
Capabilities
text_to_speech synthesizes audio from text and plays it on the user's computer. speech_to_text transcribes a .wav file into text.
How to install
from llama_index.tools.azure_speech import AzureSpeechToolSpec
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI
speech_tool = AzureSpeechToolSpec(speech_key="your-key", region="eastus")
agent = FunctionAgent(
tools=speech_tool.to_tool_list(),
llm=OpenAI(model="gpt-4.1"),
)
Requires an Azure speech resource key and region.
Who it's for
Developers building LlamaIndex agents that need to speak responses aloud or transcribe and summarize spoken audio, via Azure's speech services.
Source README
Azure Speech Tool
This tool allows Agents to use Microsoft Azure speech services to transcribe audio files to text, and create audio files from text. To see more and get started, visit https://azure.microsoft.com/en-us/products/ai-services/ai-speech
Usage
This tool has a more extensive example usage documented in a Jupyter notebook here
from llama_index.tools.azure_speech import AzureSpeechToolSpec
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI
speech_tool = AzureSpeechToolSpec(speech_key="your-key", region="eastus")
agent = FunctionAgent(
tools=speech_tool.to_tool_list(),
llm=OpenAI(model="gpt-4.1"),
)
print(await agent.run('Say "hello world"'))
print(
await agent.run(
"summarize the data/speech.wav audio file into a few sentences"
)
)
text_to_speech: Takes an input string and synthesizes audio to play on the users computerspeech_to_text: Takes a .wav file and transcribes it into text
This loader is designed to be used as a way to load data as a Tool in a Agent.
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.