Context Rot Detection

MCP service that gives AI agents self-awareness about their cognitive state.

Every long-running AI agent suffers from context rot — measurable performance degradation as the context window fills up. Research from Chroma, Stanford ("lost-in-the-middle"), and Redis confirms this is the #1 practical failure mode in production agent systems.

An agent experiencing context rot doesn't know it's degrading — it just starts making worse decisions. This tool gives agents real-time visibility into their own cognitive health.

Features

- Health score (0–100) based on token utilization, retrieval accuracy, and session fatigue
- Model-specific degradation curves for 15+ curated models (Claude, GPT, Gemini, o-series)
- Auto-resolves any HuggingFace model — pass a repo ID like meta-llama/Llama-3.1-70B and the context window is detected automatically, with results cached in SQLite
- Lost-in-the-middle risk scoring based on Stanford research
- Tool-call burden and session fatigue analysis
- Actionable recovery recommendations — compact context, offload to memory, checkpoint, break into subtasks
- Per-agent health history tracking (SQLite)
- Service-wide utilization statistics

Quick Start

npx (zero install)

npx context-rot-detection

npm (global install)

npm install -g context-rot-detection
context-rot-detection

MCP Client Configuration

Claude Code

Add to .mcp.json in your project root:

{
  "mcpServers": {
    "context-rot-detection": {
      "command": "npx",
      "args": ["-y", "context-rot-detection"],
      "env": {
        "HEALTH_HISTORY_DB": "./health.db"
      }
    }
  }
}

Claude Desktop

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "context-rot-detection": {
      "command": "npx",
      "args": ["-y", "context-rot-detection"],
      "env": {
        "HEALTH_HISTORY_DB": "/path/to/health.db"
      }
    }
  }
}

Docker

{
  "mcpServers": {
    "context-rot-detection": {
      "command": "docker",
      "args": [
        "run", "-i", "--rm",
        "-v", "context-rot-data:/data",
        "ghcr.io/milos-product-maker/context-rot-detection:latest"
      ]
    }
  }
}

Configuration

| Environment Variable | Description | Default |
|---|---|---|
| HEALTH_HISTORY_DB | Path to SQLite database for health history. Use :memory: for ephemeral storage. | :memory: |
| LOG_FILE | Path to append structured JSON log lines. Omit to disable file logging. | (none) |

Tools

check_my_health

Analyze the current context window health. Call this periodically during long sessions or before critical decisions.

Parameters:

| Parameter | Type | Required | Description |
|---|---|---|---|
| token_count | integer | Yes | Current estimated token count in context window |
| model | string | No | LLM model identifier — a curated name (e.g., claude-opus-4, gpt-4o), a HuggingFace repo ID (e.g., meta-llama/Llama-3.1-70B), or any string (falls back to conservative defaults) |
| session_duration_minutes | integer | No | How long this session has been running |
| tool_calls_count | integer | No | Number of tool calls made in this session |
| context_summary | string | No | Brief summary of current task and recent actions |
| agent_id | string | No | Unique agent identifier for history tracking |

Example response:

{
  "health_score": 62,
  "status": "warning",
  "token_utilization": {
    "current": 155000,
    "max_effective": 170000,
    "percentage": 91.2,
    "danger_zone_starts_at": 170000
  },
  "quality_estimate": {
    "retrieval_accuracy": "degrading",
    "middle_content_risk": "high",
    "estimated_hallucination_risk": "moderate"
  },
  "session_fatigue": {
    "tool_call_burden": "moderate",
    "session_length_risk": "low",
    "recommendation": "Consider breaking into sub-tasks if complexity increases."
  },
  "recommendations": [
    {
      "priority": "high",
      "action": "compact_context",
      "reason": "You are approaching the effective quality threshold. Summarize older context and remove completed task details.",
      "estimated_quality_gain": 15
    },
    {
      "priority": "high",
      "action": "offload_to_memory",
      "reason": "High risk of lost-in-the-middle effect. Store critical information to external memory before it is effectively lost.",
      "estimated_quality_gain": 8
    }
  ]
}

get_health_history

Retrieve health check history for a specific agent.

Parameters:

| Parameter | Type | Required | Description |
|---|---|---|---|
| agent_id | string | Yes | Unique agent identifier |
| limit | integer | No | Max records to return (default: 20, max: 100) |

get_service_stats

Get service-wide utilization statistics. No parameters required.

Returns total calls, unique agents, average health score, model distribution, status distribution, and recent activity (last hour / last 24h).

Supported Models

| Model | Max Tokens | Danger Zone | Middle-Loss Risk |
|---|---|---|---|
| claude-opus-4-5 | 200K | 175K | Low |
| claude-opus-4 | 200K | 170K | Low |
| claude-sonnet-4 | 200K | 165K | Low |
| claude-3.7-sonnet | 200K | 160K | Low–Medium |
| claude-3.5-sonnet | 200K | 152K | Medium |
| claude-haiku-3.5 | 200K | 130K | Medium |
| gpt-4.1 | 1M | 500K | Medium |
| gpt-4.1-mini | 1M | 450K | Medium |
| gpt-4o | 128K | 105K | Medium |
| gpt-4o-mini | 128K | 95K | Medium–High |
| o3 | 200K | 160K | Low–Medium |
| o4-mini | 200K | 150K | Medium |
| gemini-2.5-pro | 1M | 600K | Medium |
| gemini-2.5-flash | 1M | 520K | Medium–High |
| gemini-2.0-flash | 1M | 500K | High |

HuggingFace Auto-Resolution

Any model string containing / is treated as a HuggingFace repo ID. The server fetches config.json from the repo, extracts the context window size (max_position_embeddings, n_positions, or max_seq_len), and generates a conservative degradation profile:

- 65% of max tokens → degradation onset
- 80% of max tokens → danger zone

Results are cached in SQLite — subsequent lookups are instant.

model: "meta-llama/Llama-3.1-70B"       → 131K context, danger at 105K
model: "mistralai/Mistral-7B-v0.1"      → 32K context, danger at 26K
model: "mosaicml/mpt-7b"                → 65K context, danger at 52K

If the fetch fails (network error, gated model, missing config), the server falls back silently to conservative defaults.

Fallback

Any unrecognized model string without / falls back to conservative defaults (128K max, 100K danger zone).

How It Works

The health score is a weighted composite of four signals:

| Signal | Weight | Source |
|---|---|---|
| Token utilization quality | 40% | Model-specific sigmoid degradation curve |
| Retrieval accuracy | 25% | Base accuracy minus lost-in-the-middle penalty |
| Tool-call burden | 20% | Compounding quality loss after 10+ tool calls |
| Session length | 15% | Time-based fatigue heuristic |

The degradation curves are derived from empirical research:
- Chroma: Context Rot — quality degrades around 147K–152K tokens on 200K models
- Stanford: Lost in the Middle — retrieval accuracy drops for information in the middle of the context window
- Redis: Context Rot — compounding degradation effects in long-running agents

Development

git clone https://github.com/milos-product-maker/context-rot-detection.git
cd context-rot-detection
npm install
npm run dev        # Run with tsx (hot reload)
npm test           # Run unit tests
npm run build      # Compile TypeScript

Testing with MCP Inspector

npx @modelcontextprotocol/inspector node dist/index.js

License

MIT