PromptThin

Reduce LLM API costs through caching, compression, and smart routing. Zero code changes.

PromptThin is a transparent proxy that sits between your AI agents and LLM providers. Two environment variables and you're done — every API call gets five compounding savings routes applied automatically.

Your app ──→ PromptThin ──→ OpenAI / Anthropic / Gemini / Groq

PromptThin saves tokens whenyou control the API call— your own code, AI agents, or a self-hosted chat UI. It doesnotintercept calls made by managed chat interfaces like claude.ai, ChatGPT, or similar products; those platforms call LLM APIs internally and cannot be proxied.

Tip for heavy claude.ai users:If you're hitting usage quota limits in the claude.ai chat, the fix is to use a self-hosted UI likeOpen WebUIorLibreChatpointed at the Anthropic API through PromptThin. You get the same chat experience with compression and caching reducing every turn's token cost.

The semantic cache only ever stores successful, well-formed answers — seeCache correctnessbelow — and skips multimodal (image) requests by default — seeVision and image requests. Need part of a prompt to survive compression untouched? SeeProtecting parts of a prompt from compression.

Sign up atpromptthin.tech— verify your email, then start your 7-day free trial (no charge for 7 days).

curl -X POST https://promptthin.tech/auth/register \ -H "Content-Type: application/json" \ -d '{"email": "you@example.com", "password": "yourpassword"}'

# OpenAI curl -X POST https://promptthin.tech/keys/openai \ -H "X-API-Key: ts_your_key" \ -H "Content-Type: application/json" \ -d '{"api_key": "sk-your-openai-key"}' # Anthropic curl -X POST https://promptthin.tech/keys/anthropic \ -H "X-API-Key: ts_your_key" \ -H "Content-Type: application/json" \ -d '{"api_key": "sk-ant-your-anthropic-key"}' # Gemini curl -X POST https://promptthin.tech/keys/gemini \ -H "X-API-Key: ts_your_key" \ -H "Content-Type: application/json" \ -d '{"api_key": "AIza-your-gemini-key"}' # Groq curl -X POST https://promptthin.tech/keys/groq \ -H "X-API-Key: ts_your_key" \ -H "Content-Type: application/json" \ -d '{"api_key": "gsk_your-groq-key"}'

# .env — two lines, no other changes needed OPENAI_BASE_URL=https://promptthin.tech/v1 OPENAI_API_KEY=ts_your_key

curl -X POST https://promptthin.tech/v1/chat/completions \ -H "X-API-Key: ts_YOUR_API_KEY_HERE" \ -H "Content-Type: application/json" \ -d '{ "model": "gemini-2.5-flash", "messages": [{"role": "user", "content": "Hello"}], "max_tokens": 300 }'

modelcan be any supported model name (gpt-,claude-,gemini-,llama-/mixtral-/gemma-) — PromptThin infers the provider and translates the request/response shape automatically, so the same OpenAI-style call works across all four providers.

from openai import OpenAI client = OpenAI( base_url="https://promptthin.tech/v1", api_key="ts_your_key", # sent as Authorization: Bearer ts_your_key ) response = client.chat.completions.create( model="gemini-2.5-flash", messages=[{"role": "user", "content": "Hello"}], max_tokens=300, )

This is the same pattern used throughout this README's integration examples — nodefault_headersneeded. If you'd rather use the dedicated header instead, that also works:

client = OpenAI( api_key="dummy", # required by the SDK but unused when X-API-Key is set base_url="https://promptthin.tech/v1", default_headers={"X-API-Key": "ts_YOUR_API_KEY_HERE"}, )

Note onAuthorization:this header serves a second purpose beyond carrying yourts_key — it's also how you pass aproviderkey directly in pass-through mode (seeWhat if I want to pass my provider key directly?in the FAQ below). PromptThin distinguishes the two by prefix:ts_is treated as your account key, whilesk-,sk-ant-,AIza, andgsk_are treated as provider keys and used directly for that request.

from openai import OpenAI client = OpenAI( base_url="https://promptthin.tech/v1", api_key="ts_your_key", ) response = client.chat.completions.create( model="gpt-4o", messages=[{"role": "user", "content": "Hello!"}] )

import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://promptthin.tech/v1", apiKey: "ts_your_key", });

import anthropic client = anthropic.Anthropic( base_url="https://promptthin.tech", api_key="ts_your_key", )

import Anthropic from "@anthropic-ai/sdk"; const client = new Anthropic({ baseURL: "https://promptthin.tech", apiKey: "ts_your_key", });

from langchain_openai import ChatOpenAI llm = ChatOpenAI( base_url="https://promptthin.tech/v1", api_key="ts_your_key", model="gpt-4o", )

config_list = [{ "model": "gpt-4o", "base_url": "https://promptthin.tech/v1", "api_key": "ts_your_key", }]

CrewAI / any OpenAI-compatible framework

OPENAI_BASE_URL=https://promptthin.tech/v1 OPENAI_API_KEY=ts_your_key

import { createOpenAI } from "@ai-sdk/openai"; const openai = createOpenAI({ baseURL: "https://promptthin.tech/v1", apiKey: "ts_your_key", });

import litellm litellm.api_base = "https://promptthin.tech/v1" litellm.api_key = "ts_your_key"

- OpenAI API Base URL:https://promptthin.tech/v1
- API Key:ts_your_key

PromptThin infers the provider from the model name automatically:

Use thePOST /predict-savingsendpoint to get a cost estimate before making a real LLM call — no tokens billed, no LLM call made:

curl -X POST https://promptthin.tech/predict-savings \ -H "X-API-Key: ts_your_key" \ -H "Content-Type: application/json" \ -d '{ "model": "gpt-4o", "provider": "openai", "messages": [ {"role": "user", "content": "your long prompt here..."} ] }'

{ "original_tokens": 4200, "estimated_tokens_after_savings": 2100, "estimated_cost_original": 0.0105, "estimated_cost_after_savings": 0.0013, "estimated_saving": 0.0092, "saving_percent": 87.5, "recommendation": "proceed" }

PromptThin supports both Streamable HTTP (recommended) and SSE transports.

{ "mcpServers": { "promptthin": { "command": "cmd", "args": [ "/c", "npx", "mcp-remote@latest", "https://promptthin.tech/mcp", "--header", "X-API-Key: ts_your_key" ] } } }

On Mac/Linux, replace"command": "cmd"and remove"/c"— use"command": "npx"directly.

Whatproxy_chatis (and isn't):proxy_chatroutes a single outbound LLM call through PromptThin from within an AI assistant's response — for example, when you ask Claude to "use GPT-4 to summarise this file." It doesnotproxy the main conversation between you and a managed chat interface like claude.ai or ChatGPT — those platforms control their own API calls internally and cannot be intercepted. PromptThin saves tokens whereyoucontrol the API call: your own code, agents, or self-hosted chat UIs.

# 1. Check savings estimate first (free) estimate = call_tool("proxy_predict", model="gpt-4o", messages=messages) # → "87% saving — compression + routing to gpt-4o-mini" # 2. Send through PromptThin (savings applied automatically) response = call_tool("proxy_chat", model="gpt-4o", messages=messages) # → Returns answer + "[PromptThin] Tokens: 420 in / 85 out"

The semantic cache fingerprints a request from thetextportion of its messages only — image content blocks are never embedded. This means two requests with identical text but different images would otherwise hash to the same cache key and risk returning a cached answer about the wrong image.

To prevent this, PromptThinskips the semantic cache by default for any request containing image content— covering OpenAI/Anthropic-style image blocks (image_url,image,input_image) and Gemini-style inline/file image parts, in any message of the conversation, not just the latest one.

All other savings routes (compression, pruning, routing, thinking budget) are unaffected and still apply normally to vision requests.

If you have a workload where this is safe — for example, the image is decorative and the answer is fully determined by the text — you can opt back in for a single request:

curl -X POST https://promptthin.tech/v1/chat/completions \ -H "X-API-Key: ts_your_key" \ -H "X-Cache-Control: force-image-cache" \ -H "Content-Type: application/json" \ -d '{ ... }'

This is a per-request override, not a setting — each request containing images still needsforce-image-cacheexplicitly to be cached.

Protecting parts of a prompt from compression

Prompt compression (Route B) compresses the entire text of your last user message. If a part of that message must survive byte-for-byte — JSON you're going to parse, code, an exact template, anything format-sensitive — wrap it in markers instead of disabling compression for the whole message:

Please summarize this: <<<no-compress>>> {"id": 123, "exact": "json"} <<<end-no-compress>>>

- Before compression runs, each<<<no-compress>>>...<<<end-no-compress>>>block is extracted and replaced with a unique placeholder token.
- LLMLingua-2 compresses the remaining text, with the placeholder tokens hinted as force-preserved.
- After compression, PromptThin verifies every placeholder token survived intact. If even one was split, stripped, or altered by the tokenizer, theentire compression result for that message is discardedand the original uncompressed message is sent instead — this guarantees the protected content is never silently corrupted, at the cost of losing compression savings on that one message.
- If the verification passes, the placeholders are replaced back with the original protected text and the markers are removed from the final message.

Malformed markers (e.g. unmatched start/end tags) are treated as plain text — the message compresses normally without raising an error.

This is a finer-grained alternative toX-Compress-Control: no-compress, which disables compression for the whole request rather than just a portion of one message.

The semantic cache is only ever populated with responses that PromptThin can verify are well-formed. Before any response is written to the cache, it must pass all of the following checks:

- No transport or provider error— the upstream call must return HTTP 200. Timeouts, gateway errors, and provider-side error payloads are never cached.
- Non-empty, substantive content— responses with empty or near-empty text (e.g. a thinking model that returned nothing because its reasoning budget consumed the entire output) are rejected.
- No bad finish reason— provider-specific signals that the response was cut short or blocked are checked: OpenAI/Groq content-filter stops, GeminiSAFETY/RECITATION/OTHER/BLOCKLISTfinish reasons, and Anthropicstop_reason == "error"all skip the cache.

These checks catcherrors and malformed responses, not factual correctness — PromptThin has no way to verify whether a fluent, well-formed answer is actuallyright. If you're working with prompts where you don't want a possibly-imperfect answer cached for future similar requests by anyone, sendX-Cache-Control: no-cacheon that request. It skips both the cache read and the cache write, so that response is never reused.

For multimodal requests specifically, seeVision and image requestsabove — those are skipped by default regardless of response quality, because the risk there is a cache-key collision, not a bad response.

client = OpenAI( base_url="https://promptthin.tech/v1", api_key="ts_your_key", default_headers={"Authorization": "Bearer sk-your-openai-key"}, )

Enhances AI reasoning by providing a structured thinking environment.