Together.ai Mcp
About
MCP server exposing Together AI inference endpoints (chat, image, vision, embeddings) as tools for Claude Desktop, Cursor, VS Code, and any other MCP-compatible client. Fixes silent empty-response bug in reasoning models.
Details
- Author
- Leonfinn
- Downloads
- 281
- Categories
- AI
Jump to
- Chat completions with any Together AI text or reasoning model
- Correctly handles reasoning models that write output to non-standard fields
- Image generation via FLUX, SDXL models, saved as PNG files
- Vision analysis using Llama 3.2 Vision or Qwen 2.5 VL
- Embeddings generation for RAG/retrieval pipelines
- Uses direct fetch calls for vision and image generation to avoid SDK limitations
Setting up with Highlight
This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:
- Download and install Highlight from highlightai.com/download
- Navigate to the plugins tab and select "Add Custom Plugin"
-
Configure the plugin with the settings below
Plugin Name
Together.ai McpCommand (node, npx, python, etc.)Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
- Enable "Start Automatically" if you want the plugin to start when Highlight launches
From the repository
Install Node.js 18+, obtain a Together AI API key, clone the repository, run npm install, copy .env.example to .env, add your API key, then configure the server in your MCP client (e.g., Claude Desktop) by adding a together-ai entry to the JSON config with the path to index.js and required environment variables.
Claude Desktop / Cursor
Paste into your MCP client config file to install this server.
{
"mcpServers": {
"together.ai mcp": {
"together-ai": {
"command": "node",
"args": [
"/absolute/path/to/together-ai-mcp/index.js"
],
"env": {
"TOGETHER_API_KEY": "your_api_key_here",
"IMAGE_OUTPUT_DIR": "/path/to/save/images"
}
}
}
}
}McpServers
{
"together-ai": {
"command": "node",
"args": [
"/absolute/path/to/together-ai-mcp/index.js"
],
"env": {
"TOGETHER_API_KEY": "your_api_key_here",
"IMAGE_OUTPUT_DIR": "/path/to/save/images"
}
}
}together-ai-mcp
A Node.js Model Context Protocol (MCP) server that exposes Together AI's inference endpoints — chat completions, image generation, vision, and embeddings — as tools callable from Claude Desktop, Cursor, VS Code, and any other MCP-compatible client.
Why this exists
I created this MCP due to several issues I was having accessing models through Together AI.
1. Reasoning model silent empty responses
Together AI's largest reasoning models (GLM-5, Qwen3.5-397B, MiniMax M2.5, Kimi K2.5) write their chain-of-thought into non-standard response fields, and they exhaust the OpenAI SDK's default token budget before producing a final answer.
Two problems compound each other:
Token budget exhaustion. The OpenAI SDK sets a default max_tokens of 2048. For reasoning models, this budget is consumed entirely by the thinking phase — message.content is never populated. You get charged for tokens, no error is raised, and the response is silently empty.
Fragmented response fields. Different model families on Together AI write their output to different fields:
| Field | Used by |
|---|---|
| message.content | Standard models; Qwen (inline <think> tags) |
| message.reasoning_content | DeepSeek-style format |
| message.reasoning | Together AI format (GLM-5, MiniMax, Kimi) |
Any code that only reads message.content — or even message.content \|\| message.reasoning — silently returns an empty string for some models.
// Broken — misses reasoning_content (DeepSeek format):
const text = message.content || message.reasoning || '';
// Fixed — covers all Together AI reasoning model formats:
const text = message.content || message.reasoning_content || message.reasoning || '';
The default max_tokens is raised to 8192 to give reasoning models enough budget to complete their chain of thought before producing a final answer.
2. Vision model failures
Using the OpenAI SDK's chat.completions.create() for vision requests fails silently against Together AI's vision API. Together AI requires stream: false to be set explicitly; the SDK may not send it. When it does fail, the SDK error contains no response body, making the root cause invisible.
// Broken — SDK may omit stream:false; errors are opaque:
const response = await openai.chat.completions.create({ model, messages });
// Fixed — raw fetch, explicit stream:false, full error body in exception:
const response = await fetch('https://api.together.xyz/v1/chat/completions', {
method: 'POST',
headers: { Authorization: Bearer ${apiKey}, 'Content-Type': 'application/json' },
body: JSON.stringify({ model, messages, max_tokens, stream: false }),
});
if (!response.ok) {
const body = await response.text();
throw new Error(Vision API error ${response.status}: ${body.slice(0, 200)});
}
---
Features
- Chat completions — any Together AI text or reasoning model, with full prompt and multi-turn message support
- Reasoning model support — correctly handles GLM-5, Qwen3.5-397B, MiniMax M2.5, Kimi K2.5 (see above)
- Image generation — FLUX.1-dev, FLUX.1-schnell, Stable Diffusion XL; images saved to disk
- Vision — analyse images via Llama 3.2 Vision or Qwen 2.5 VL
- Embeddings — generate vectors for RAG/retrieval pipelines via BGE and Snowflake Arctic models
---
Installation
Prerequisites
- Node.js 18+
- A Together AI API key — get one at api.together.ai
Setup
```bash
git clone https://github.com/your-username/together-ai-mcp
cd together-ai-mcp
npm install
cp .env.example .env
Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
