Arkheia Hallucination Detection

by arkheiaai

Not rated
GitHub

About

Detect fabrication and hallucination in any LLM output. Score responses from GPT-4o, Claude, Gemini, Llama and 30+ models. Free tier included.

Details

Author
arkheiaai
Categories
Other, AI, Developer Tools

Setup

Install Arkheia Hallucination Detection in your MCP client (Claude Desktop, Cursor, Windsurf, and others).

Repository: https://github.com/arkheiaai/arkheia-mcp

Follow the installation instructions in the repository README, then restart your MCP client.

Detect fabrication and hallucination in any LLM output. Score responses from GPT-4o, Claude, Gemini, Llama and 30+ models. Free tier included.

Arkheia MCP Server — Fabrication Detection for AI Agents

Arkheia screens model responses for fabrication using behavioural fingerprinting. Works with Claude, GPT, Gemini, Grok, Llama, Mistral, and 30+ other models. One tool call. Real-time risk scoring.

Free tier: 1,500 detections/month. No credit card.

The installer sets up a Python environment, clones the server, and configures everything. Takes about 60 seconds.

- Node.js 18+
- Python 3.10+
- An API key (free — see below)

curl -X POST https://arkheia-proxy-production.up.railway.app/v1/provision \ -H "Content-Type: application/json" \ -d '{"email": "you@example.com"}'

Save the key. You won't see it again.

{ "mcpServers": { "arkheia": { "command": "python", "args": ["-m", "mcp_server.server"], "cwd": "~/.arkheia/mcp", "env": { "PYTHONPATH": "~/.arkheia/mcp", "ARKHEIA_API_KEY": "ak_live_your_key_here" } } } }

On Windows, replace~/.arkheia/mcpwithC:/Users/YourName/.arkheia/mcp.

Add to yourclaude_desktop_config.json(macOS:~/Library/Application Support/Claude/, Windows:%APPDATA%\Claude\):

{ "mcpServers": { "arkheia": { "command": "python", "args": ["-m", "mcp_server.server"], "cwd": "~/.arkheia/mcp", "env": { "PYTHONPATH": "~/.arkheia/mcp", "ARKHEIA_API_KEY": "ak_live_your_key_here" } } } }

Arkheia works with any agent that supports MCP tools — Cursor, Windsurf, Cline, or your own orchestrator. The configuration pattern is the same: point the MCP server at~/.arkheia/mcpwith your API key.

Restart your agent after adding the configuration.

For local Enterprise Proxy audit-log retrieval, setARKHEIA_PROXY_AUTH_TOKENto a proxy JWT session token. This is distinct fromARKHEIA_API_KEY: the MCP client sends it only toARKHEIA_PROXY_URLasAuthorization: Bearer ...for protected local endpoints such asarkheia_audit_log.

"Use arkheia_verify to check this response: HTTP/2 introduces multiplexing which allows multiple requests over a single TCP connection."

You should see aLOWrisk result — that's a truthful response.

"Use arkheia_verify to check this response: The Kafka 4.1 ConsumerLease API introduces a lease-based partition ownership model that replaces the traditional rebalance protocol."

You should see aHIGHrisk result — the Kafka 4.1 ConsumerLease API doesn't exist. Arkheia caught it.

These exercise different detection scenarios. Run them through your agent to see how it handles each:

- "Use arkheia_verify on: Docker caches each Dockerfile layer. Unchanged layers are reused. This is why COPY order matters."
- "Use arkheia_verify on: PostgreSQL uses MVCC to handle concurrent reads and writes without locking rows."
- "Use arkheia_verify on: A JWT has three parts: header, payload, and signature, each base64-encoded."

- "Use arkheia_verify on: The GraphQL Federation 3.0 EntityBridge directive enables cross-subgraph entity resolution without shared key fields."
- "Use arkheia_verify on: Docker BuildKit 3.0's SnapshotDelta feature reduces layer push size by transmitting only changed filesystem blocks."
- "Use arkheia_verify on: PostgreSQL 18 introduced REINDEX PARALLEL which coordinates workers to avoid lock contention on shared catalogs."

Try this — it's genuinely interesting:

"You now have access to arkheia_verify for fabrication detection. How would you use this to improve the quality of your own outputs? Try verifying one of your own responses."

Your agent will explore the tool, test it on its own output, and tell you what it found. This is the best way to see how detection integrates into a real workflow.

Copy this into your project'sCLAUDE.md(or equivalent agent instruction file) to make fabrication detection automatic across every conversation:

# Fabrication Detection This project uses Arkheia for runtime fabrication detection. The arkheia_verify MCP tool is available in every conversation. ## Verification Protocol Before presenting any substantive response to the user: 1. Call arkheia_verify with the model name, prompt, and response 2. Check the risk field in the result ### Risk Handling - LOW: Present normally - MEDIUM: Present with caveat — "Detection flagged medium confidence. Key claims should be verified." - HIGH: Do not present as-is. Investigate the specific claims. If unverifiable, regenerate or escalate. ### Sub-Agent Outputs When spawning background agents or parallel workers: - Verify each agent's output independently before merging - A HIGH risk from any agent blocks the merge until investigated - Log all detection results for audit ### What NOT to Do - Do not skip verification because the response "looks correct" - Do not suppress HIGH findings — the user needs to know - Do not retry the same prompt expecting a different risk score

A ready-to-use template file is available atCLAUDE_MD_TEMPLATE.md.

If you use multiple AI agents (Claude + Codex, Gemini + Grok, etc.), detection becomes your quality gate:

1. Draft agent generates a response 2. arkheia_verify screens the response → risk score 3. If LOW: accept 4. If MEDIUM: second agent reviews the specific claims 5. If HIGH: regenerate with a different model, or flag for human review

This catches fabrication that individual agents miss. The draft agent is confident. The detection layer is objective. The review agent has context. Together they produce higher quality output than any single agent.

- OpenAI:GPT-4o, GPT-5.4, GPT-5-Codex family
- Anthropic:Claude Opus 4.6, Sonnet 4.6, Haiku 4.5
- Google:Gemini 2.5 Pro/Flash, Gemini 3 Pro Preview
- xAI:Grok 4, Grok 4 Fast, Grok Code Fast
- Local:Qwen2 72B, Phi4, Mixtral, CodeLlama, Falcon
- Others:Kimi K2.5, Ouro

If your model isn't listed,let us knowand we'll characterise it. We add new models regularly.

The MCP server provides the richest detection because it captures the full inference signal during model calls. If you have a specific workflow where you need to call the detection API directly (CI/CD pipelines, custom orchestrators, batch processing), the REST endpoint is available:

POST https://arkheia-proxy-production.up.railway.app/v1/detect

Direct API calls without inference data provide structural analysis only. For full behavioural fingerprinting, use the MCP tools — they capture everything automatically. If you're building a custom integration and want full detection quality,get in touchand we'll help you set it up.

No credit card for free tier. Upgrade when you're ready.

We built this because we needed it. We run 151 AI agents in production and every one of them is screened by Arkheia.

If you're using it — whether you love it, hate it, or wish it did something different — we want to hear from you:

- GitHub Issues:https://github.com/arkheiaai/arkheia-mcp/issues— bugs, feature requests, questions
- Email:
dmurfet@arkheia.ai— anything at all

Every message is read. Every piece of feedback shapes what we build next.

- Python 3.10+
- Node.js 18+ (for npx install)
- Git

- Website:https://arkheia.ai
- GitHub:
https://github.com/arkheiaai/arkheia-mcp
- Support:
dmurfet@arkheia.ai

Hosted, Stateless & Multitenant Gemini MCP server enables AI assistants to access multimodal AI capabilities, content generation, and reasoning workflows through Google Gemini.

Queries multiple Ollama models to combine their responses, offering diverse AI perspectives on a single question.

Remote MCP for prompt injection scans, jailbreak checks, leakage review, and LLM release gates.

Consult multiple AI models, including local, cloud, and enterprise services, to get diverse perspectives on a topic.

An MCP server for AI video generation. MCP server for AI video generation. Lets Claude, ChatGPT, OpenClaw , Hermes & other agents create AI videos and publish them to YouTube, TikTok, Instagram etc..

HumanDesign.ai MCP is the official account-connected Human Design server for Claude, ChatGPT, Codex, Cursor, and VS Code.

3D Cartoon Generator & File System Tools

Generates 3D-style cartoon images using Google's Gemini AI and provides secure file system operations.

Detects bias in AI model outputs across protected characteristics — statistical parity checks, disparate impact analysis, intersectional fairness audits, and EU AI Act Article 10 compliance reporting.

Audited tarot, I Ching, and Xiao Liu Ren divination MCP server. Local seeded entropy; the model only interprets JSON output.

No reviews yet — be the first

Sign in to leave a review

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.