Kafka MCP Server
About
An MCP server for Apache Kafka, enabling LLMs to perform Kafka operations like producing and consuming messages.
Details
- License
- Apache-2.0 license
Explore
- Kafka Integration: Implementation of common Kafka operations via MCP
- Security:
- Support for SASL (PLAIN, SCRAM-SHA-256, SCRAM-SHA-512) and TLS authentication
- OAuth 2.1 authentication for HTTP transport (Native and Proxy modes)
- Support for Okta, Google, Azure AD, and HMAC providers
- Flexible Transport: STDIO for local clients, HTTP for remote access
- Error Handling: Error handling with meaningful feedback
- Configuration Options: Customizable for different environments
- Pre-Configured Prompts: Set of prompts for common Kafka operations
- Compatibility: Works with MCP-compatible LLM models
- Go 1.24 or later
- Docker (for running integration tests)
- Access to a Kafka cluster
brew install kafka-mcp-server
To update to the latest version:
brew update && brew upgrade kafka-mcp-server
claude mcp add kafka \
--env KAFKA_BROKERS=localhost:9092 \
--env KAFKA_CLIENT_ID=kafka-mcp-server \
--env MCP_TRANSPORT=stdio \
--env KAFKA_SASL_MECHANISM= \
--env KAFKA_SASL_USER= \
--env KAFKA_SASL_PASSWORD= \
--env KAFKA_TLS_ENABLE=false \
-- kafka-mcp-server
Other useful commands:
claude mcp list
Managing MCP server configurations across multiple clients can become challenging. mcpenetes is a dedicated tool that makes this process significantly easier:
go install github.com/tuannvm/mcpenetes@latest
mcpenetes apply
mcpenetes load
``
With mcpenetes, you can maintain multiple Kafka configurations (development, production, etc.) and switch between them instantly across all your clients (Cursor, Claude Desktop, Windsurf, ChatWise) without manually editing each client's configuration files.
When using HTTP transport (MCP_TRANSPORT=http), OAuth 2.1 authentication can be enabled:
| Variable | Description | Default | Required |
| :------- | :---------- | :------ | :------- |
| MCP_HTTP_PORT | HTTP server port | 8080 | No |OAUTH_ENABLED
| | Enable OAuth 2.1 authentication | false | No |OAUTH_MODE
| | OAuth mode: native or proxy | native | No |OAUTH_PROVIDER
| | Provider: hmac, okta, google, azure | okta | No |OAUTH_SERVER_URL
| | Full MCP server URL (e.g., https://localhost:8080) | - | When OAuth enabled |OIDC_ISSUER
| | OAuth issuer URL | - | When OAuth enabled |OIDC_AUDIENCE
| | OAuth audience | - | When OAuth enabled |OIDC_CLIENT_ID
| | OAuth client ID | - | Proxy mode only |OIDC_CLIENT_SECRET
| | OAuth client secret | - | Proxy mode only |OAUTH_REDIRECT_URIS
| | Comma-separated redirect URIs | - | Proxy mode only |JWT_SECRET
| | JWT signing secret | - | Proxy mode only |
For detailed OAuth setup and examples, see docs/oauth.md.
> Security Notes:
> - When using KAFKA_TLS_INSECURE_SKIP_VERIFY=true`, the server will skip TLS certificate verification. This should only be used in development or testing environments, or when using self-signed certificates.
> - OAuth is only available when using HTTP transport. STDIO transport does not support OAuth.
> - Always use HTTPS in production when OAuth is enabled.
The server exposes the following tools for Kafka interaction. For detailed documentation including examples and sample responses, see docs/tools.md.
- produce_message: Produces messages to Kafka topics
- consume_messages: Consumes messages from Kafka topics in batch operations
- list_brokers: Lists all configured Kafka broker addresses
- describe_topic: Provides comprehensive metadata for specific topics
- list_consumer_groups: Enumerates all consumer groups in the cluster
- describe_consumer_group: Provides detailed consumer group information including lag metrics
- describe_configs: Retrieves configuration settings for Kafka resources
- cluster_overview: Provides comprehensive cluster health summaries
- list_topics: Lists all topics with metadata including partition and replication information
A Model Context Protocol (MCP) server for Apache Kafka implemented in Go, leveraging franz-go and mcp-go.
This server provides an implementation for interacting with Kafka via the MCP protocol, enabling LLM models to perform common Kafka operations through a standardized interface.
Overview
The Kafka MCP Server bridges the gap between LLM models and Apache Kafka, allowing them to:
- Produce and consume messages from topics
- List, describe, and manage topics
- Monitor and manage consumer groups
- Assess cluster health and configuration
- Execute standard Kafka operations
All through the standardized Model Context Protocol (MCP).
Architecture
graph TB
subgraph "MCP Client (AI Applications)"
A[Claude Desktop]
B[Cursor]
C[Windsurf]
D[ChatWise]
end
subgraph "Kafka MCP Server"
E[MCP Protocol Handler]
F[Tools Registry]
G[Resources Registry]
H[Prompts Registry]
I[Kafka Client Wrapper]
end
subgraph "Apache Kafka Cluster"
J[Broker 1]
K[Broker 2]
L[Broker 3]
M[Topics & Partitions]
N[Consumer Groups]
end
A --> E
B --> E
C --> E
D --> E
E --> F
E --> G
E --> H
F --> I
G --> I
H --> I
I --> J
I --> K
I --> L
J --> M
K --> M
L --> M
J --> N
K --> N
L --> N
classDef client fill:#e1f5fe
classDef mcp fill:#f3e5f5
classDef kafka fill:#fff3e0
class A,B,C,D client
class E,F,G,H,I mcp
class J,K,L,M,N kafka
How it works:
1. MCP Clients (AI applications) connect to the Kafka MCP Server via stdio or HTTP transport
2. MCP Server exposes three types of capabilities:
- Tools - Direct Kafka operations (produce/consume messages, describe topics, etc.)
- Resources - Cluster health reports and diagnostics
- Prompts - Pre-configured workflows for common operations
3. Kafka Client Wrapper handles all Kafka communication using the franz-go library
4. Apache Kafka Cluster processes the actual message streaming and storage
Transport Modes:
- STDIO: Default mode, ideal for local MCP clients (Claude Desktop, Cursor, etc.)
- HTTP: Enables remote access with optional OAuth 2.1 authentication
Key Features
- Kafka Integration: Implementation of common Kafka operations via MCP
- Security:
- Support for SASL (PLAIN, SCRAM-SHA-256, SCRAM-SHA-512) and TLS authentication
- OAuth 2.1 authentication for HTTP transport (Native and Proxy modes)
- Support for Okta, Google, Azure AD, and HMAC providers
- Flexible Transport: STDIO for local clients, HTTP for remote access
- Error Handling: Error handling with meaningful feedback
- Configuration Options: Customizable for different environments
- Pre-Configured Prompts: Set of prompts for common Kafka operations
- Compatibility: Works with MCP-compatible LLM models
Getting Started
Prerequisites
- Go 1.24 or later
- Docker (for running integration tests)
- Access to a Kafka cluster
Installation
Homebrew (macOS and Linux)
The easiest way to install kafka-mcp-server is using Homebrew:
```bash
Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
