Manage Kafka with Natural Language
A natural-language MCP interface for managing Kafka topics, messages, brokers, and offsets.
Why it matters
Interact with Kafka clusters using natural language commands. This asset allows AI agents to publish and consume messages, manage topics, and monitor broker and partition information.
Outcomes
What it gets done
Publish and consume messages from Kafka topics.
Create, delete, and list Kafka topics.
Retrieve broker and partition details.
Manage consumer group offsets.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/vb-kafka-mcp | bash Capabilities
Tools your agent gets
Consume messages from Kafka topics
Publish messages to Kafka topics
List, create, delete, and describe topics in Kafka
Get broker information from Kafka cluster
Get partitions and partition offsets from Kafka
Get and reset offsets in Kafka consumer groups
Overview
kafka-mcp server
An MCP server that turns natural-language prompts into Kafka operations - publishing, consuming, topic management, and offset control. Use it when an AI agent needs to manage Kafka topics, messages, or offsets through natural language instead of custom client code.
What it does
A natural-language MCP interface for Kafka: it translates prompts like "publish message 'i am using kafka server' on the topic 'test-kafka'" or "list all topics from the kafka environment" into Kafka operations, giving AI agents control over producers, consumers, topics, brokers, partitions, and offsets through any MCP client.
When to use - and when NOT to
Use this when an AI agent needs to publish or consume Kafka messages, manage topics (list, create, delete, describe), inspect broker info, or read/reset partition and consumer-group offsets, driven entirely by natural-language prompts instead of writing Kafka client code. It is scoped to these Kafka administration and messaging operations - it is not a general data-pipeline or stream-processing engine.
Capabilities
- consumer / producer tools: consume and publish messages on topics, e.g. "Publish a message 'hello from mcp' to the topic 'my-kafka'" or "Consume 2 messages from the topic 'my-kafka'".
- topic tools: list, create, delete, and describe topics, e.g. "List all topics in the kafka cluster", "Create topic 'my-kafka'", or "Describe the topic 'my-kafka'".
- broker tool: get broker info.
- partition tools: get partitions and partition offsets.
- group_offset tools: get and reset consumer group offsets in Kafka.
How to install
Set BOOTSTRAP_SERVERS (your Kafka server address) and MCP_TRANSPORT=stdio in a .env file or as environment variables. For local development: create a virtual environment (python3 -m venv .venv), install dependencies with pip install -r requirements.txt or uv pip install -r requirements.txt, then run with python3 src/main.py (or uv run python3 src/main.py). To connect from a client such as Claude Desktop, add:
{
"mcpServers": {
"kafka-mcp": {
"command": "python3",
"args": ["/Users/I528600/Desktop/mcp/kafka-mcp/src/main.py"],
"env": {
"BOOTSTRAP_SERVERS": "localhost:9092",
"MCP_TRANSPORT": "stdio"
}
}
}
}
Who it's for
Developers and platform engineers building AI-driven workflows that need to manage Kafka topics, messages, and offsets through natural language instead of writing custom Kafka client integration code, in setups designed from the ground up to be scalable, lightweight, and suited to high-performance data operations.
Source README
Kafka MCP
Overview
The Kafka MCP Server offers efficient way to convert prompts into actions into Kafka ecosystem. It is a natural
language interface designed for agentic applications to efficiently manage Kafka operations and integrate seamlessly
with MCP Clients enabling AI driven workflows to interact with processes in Kafka. Using this MCP Server, you can ask
questions like:
- Publish message 'i am using kafka server' on the topic 'test-kafka'
- Consume the message from topic 'test-kafka'
- List all topics from the kafka environment
Features
- Natural Language Queries: Enables AI agents to query and update Redis using natural language.
- Seamless MCP Integration: Works with any MCP client for smooth communication.
- Full Kafka Support: Handles producer, consumer, topics, broker, partitions and offsets.
- Scalable & Lightweight: Designed for high-performance data operations.
Tools
This MCP Server offers various tools for Kafka:
consumer and producer tools allow to consumer and publish message on topics
topic tools allow to list, create, delete and describe topics in Kafka.
broker allows to get broker info.
partition tools allow to get partitions and partition offsets.
group_offset tools allow to get and reset offsets in Kafka.
Configurations
set the following in .env file or export manually
BOOTSTRAP_SERVERS=your_kafka_server
MCP_TRANSPORT=stdio
Local Development
Create a virtual environment
# Using venv (built-in)
python3 -m venv .venv
# Activate the virtual environment
# On Windows
.venv\Scripts\activate
# On macOS/Linux
source .venv/bin/activate
Install Dependencies
# Using pip
pip install -r requirements.txt
# Or using uv (faster)
uv pip install -r requirements.txt
Set Configurations in terminal/env
BOOTSTRAP_SERVERS=<your_kafka_url>
MCP_TRANSPORT=stdio
Run the application
python3 src/main.py
# OR
uv run python3 src/main.py
To interact with server,
Add the following configuration to your MCPO server's config.json file (e.g., in Claude Desktop):
{
"mcpServers": {
"kafka-mcp": {
"command": "python3",
"args": ["/Users/I528600/Desktop/mcp/kafka-mcp/src/main.py"],
"env": {
"BOOTSTRAP_SERVERS": "localhost:9092",
"MCP_TRANSPORT": "stdio"
}
}
}
}
Example prompts
- List all topics in the kafka cluster
- Create topic 'my-kafka' in kafka cluster
- Publish a message 'hello from mcp' to the topic 'my-kafka' in cluster
- Consume 2 messages from the topic 'my-kafka' in kafka cluster
- Describe the topic 'my-kafka'
FAQ
Common questions
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.