VinvAI

**Vinv (Vibe Inverse) runs your services, finds issues, and verifies fixes — with zero code changes.** It connects runtime traces to the source code that produced them, hands that evidence to your agent, then runs the code again to verify the fix actually works. It even uses**Thompson sampling**to decide*how much*runtime context to give the agent — because more is not always better. Python first — services and APIs. TS & Go next. No account. No API keys. No telemetry. Everything runs on your machine. Open source, Apache-2.0. Vinv works three ways — as an editor extension, a CLI, or an MCP server for any agent. They share the same engines. One click —[**install from vinv.ai**, which opens the extension directly in your chosen editor — or install straight from your editor's CLI: First run builds the engines — about 4 minutes: it compiles the Rust index and fetches a one-time ~500 MB local embedding model (](https://vinv.ai/#install)[uvand](https://docs.astral.sh/uv/)[Rustrequired). First trace lands about a minute after that; everything after is seconds. ``` `pip install vinv # every engine as a console script # or run one with zero install: uvx --from vinv exerciser campaign <repo> --budget 20` ``` Give Claude Code, Cursor, or any MCP client Vinv's tools. Install the engines above, then register the unified server —**one global config**; it finds your open workspace automatically via MCP roots: Other clients: add`{ "command": "npx", "args": ](https://rustup.rs)["-y", "vinv-mcp"] }`under`mcpServers.vinv`in the client's MCP config. See[`vinv-mcp`— 16 tools: semantic search, dead code, fault localization, runtime values/slices/coverage, and the verify/optimize loop. ``` `git clone https://github.com/VinvAI/VinvAI ~/.vinv/engines && cd ~/.vinv/engines && ./install.sh` ``` ``` `git clone https://github.com/VinvAI/VinvAI $HOME\.vinv\engines; cd $HOME\.vinv\engines; .\install.ps1` ``` **Run · Test · Find — then Prove.**Point Vinv at a Python repo; it does the rest — no code changes, no API keys. - **🏃 Run**— brings every service in your repo up under tracing with zero edits, capturing timings, arguments, return values and call trees from the real run. - **🧪 Test**— drives real requests through every endpoint (valid, boundary, negative, authenticated) and banks each response as a regression case. - **🔎 Find**— surfaces what actually broke or slowed down — server errors, crashes, latency hotspots and dead code — each tied to the exact source line. - **✅ Prove**— hands that evidence to the agent you already use (Claude Code, Cursor, Copilot…), then verifies its fix against acceptance tests written*before*the fix that it never sees. A "faster" change that alters any output is auto-reverted. Your agent is the only LLM — no new bill, no model picker, no provider keys. Everything runs on your machine. 84% of developers now use or plan to use AI coding tools. More of them**actively distrust**the output (46%) than trust it (33%) — and distrust nearly doubled in a year (](https://www.npmjs.com/package/vinv-mcp)[Stack Overflow 2025, 49k developers). You know why: the agent edits the wrong handler, invents return shapes, then grades its own homework while the server won't even start. Or it gets stuck — test fails, agent edits the same function, test fails the same way, agent edits it again, burning your context window on "let me verify." Anthropic's own research documents agents "stuck in loops, repeating the same failed approach" when they lack codebase context. Both failures have one root cause:**the agent has never watched your code run.**It argues from static text. The industry automated*writing*and left*proving*entirely manual. Vinv automates the proving — and only then the finding and the fixing. Receipts first — then how the loop produces them. Vinv found**four bugs and one performance problem**in](https://survey.stackoverflow.co/2025/ai/)[fastapi/full-stack-fastapi-template(~44k★). Same five issues, same prompts, Vinv grading every run: One trial per condition — a**demonstration, not a benchmark**. Blind, the commodity model scored zero. Hand it the failing frame, the caller chain, and the real argument values, and it beats a stronger model guessing from static code.**The evidence is what moved, not the weights.** On that same pristine template the optimization loop later detected — from live traces alone — that the app's default database pool makes requests**queue for connection checkouts**under concurrent load, dispatched the pool-sizing fix, and proved it: sustained-load median**75.6ms → 41.2ms, 45.4% faster (95% CI ](https://github.com/fastapi/full-stack-fastapi-template)[36.3%, 45.8%])**, responses byte-identical. Two earlier attempts whose measurement windows couldn't certify the win were**auto-reverted**— the accept landed only when the evidence did. ### Same discipline, upstream on Hugging Face Pointed at[huggingface/smolagents(~28.5k★) — a public Apache-2.0 agent framework, no affiliation — the allocation loop found and proved a fast-path in`sanitize_for_rich`. Benchmarked with`tracemalloc`on a realistic 4 KB log line: transient per-call allocation**36.27 KB → 0.00 KB (~37,137× less)**; end-to-end on`log_task`,**~615.7 KB → ~125 B**across 3 calls; output byte-identical across**2,015**inputs. Upstream as](https://github.com/huggingface/smolagents)[**PR #2572**. That is*how come after it*: oracles find the waste, your agent proposes the edit, paired-bootstrap + byte-identical replay decide accept or revert, and only then does anything go upstream. The rest of this README is the machinery behind those receipts. The**Test**stage above isn't one tester — it's a set of oracles, each hunting a different class of defect, all writing into the same findings and the same fix-dispatch path. You never need to think about them to use Vinv; open this if you want the full list and how the budget is spent. **The dispatcher is real, and it is a bandit.**`exerciser campaign`allocates**one budget**across every*armed*oracle by Thompson sampling over`(target × technique × oracle)`— rather than driving each one exhaustively. Cost is*measured*(wall-clock normalized to probe-equivalents plus subprocesses spawned), so an oracle that takes forty seconds to find what another finds in one loses. Credit is paid**once per defect signature**, within a run and across runs, so a deterministic oracle can't re-earn credit for the same bug forever. Posteriors persist in`campaign.json`—*which technique pays on your repo*is learned. The campaign dispatches**six**oracles (`default_runners`): crash/function, differential, fault, concurrency, HTTP, and environment — each*armed*only when it applies to your repo (no`--base-url`and the HTTP oracle stays dark; no boundaries and the fault oracle does). The**dead-code**and**runtime-analysis**families (leaks, hotspots, cache candidates) and the**golden-I/O baselines**in the table below run from the editor and the regression path, not this budget loop. Unverified code runs behind a**containment ladder**: a kernel-enforced OS sandbox (`sandbox-exec`/`bwrap`/`unshare`) where the host offers one, otherwise a process shim — always with a disposable repo copy, redirected`HOME`/`TMPDIR`, blocked network and subprocess spawning. Tier is decided by a*probe*that verifies a write outside the root really failed, never by a binary being on`PATH`. Postgres, Redis and S3 are substituted**inside**the jail so code that needs them runs instead of failing to connect. Static tools (Vulture,`deadcode`, Knip, ts-prune) can only prove*"nothing statically references this."*They can't see dynamic dispatch, feature flags, registries or environment drift — so they emit candidates a human has to adjudicate, which is why the cleanup never happens. **"No capture ever executed this. Here is what still references it, here is the traced neighbourhood it would wire back into, and here is what your agent thinks it is."** The unit is a**section**— a connected island of untraced symbols — not a lint row, because dead code is almost never one function. Each section gets: - **Reachability evidence**—`REACHED FROM LIVE CODE`(executing code references it; the path was never taken — usually a guard or an unshipped feature) versus`NO REFERENCES`at all. Opposite verdicts, and a filtered canvas can't tell you which. - **The live neighbourhood**— Personalized PageRank seeded at the section's symbols over the code graph, keeping the highest-mass**traced**symbols. HippoRAG's retrieval idea, run over a graph Vinv already has. That's the place an integration would wire into. - **Your agent's verdict**—`integrate`·`reimagine`·`delete`·`keep`·`unclear`, each with what it does, why nothing reaches it, and what breaks if it's removed. Sections travel five to a prompt so the agent can say*"this is the older copy of the section below"*— a judgment a per-section run structurally cannot reach. Two things it refuses to do: call anything dead with**no trace on disk**(with zero captures every symbol is untraced, so the list would be your codebase), and drop anything silently — both caps are recorded as lineage, so "12 sections" is distinguishable from "12 is the cap and 300 were dropped". And the second-order reason to care:**dead code makes your coding agent worse.**Every unused module competes for the context window, burns tokens, and offers wrong patterns to copy. Static scanners guess. Vinv records a real run and ranks what failed, what never executed, and what was slow. No "possible issue" — every finding names a symbol, a line, and the trace behind it. ``` `# with a service up — Auto-Pilot's path: plan every endpoint, then drive them exerciser plan <repo> --base-url http://127.0.0.1:PORT && exerciser run <repo> --base-url http://127.0.0.1:PORT # allocate budget across armed oracles (HTTP + the rest that apply to that run) exerciser campaign <repo> --budget 20` ``` ### 02 FIX — through the agent you already pay for Findings become evidence packs and go to your own agent — Claude Code, Codex, Cursor, Gemini CLI, Copilot Chat, Windsurf Cascade.**Your agent is Vinv's only LLM.**No new bill, no model picker, no provider keys. The pack is composed from a**context graph**: your code, your traces, and the metrics derived from them, joined on the exact function that handled each request. The artefacts are commodities; the join is not. Replayed start. Live port. Acceptance tests authored**before**the fix, stored outside the workspace under an opaque token, and required to fail deterministically*twice*on the pre-fix code — the SWE-bench fail-to-pass discipline. A test that passes on broken code is discarded. Around that: a static pre-gate (every changed`.py`must`ast.parse`before a test budget is spent), a deterministic**anti-cheat diff audit**over the snapshot ref plus untracked files (test edits,`except`swallows, interpreter shadow modules,`.vinv`tampering — hard flags block eligibility outright), a**bounded LLM judge**that can push toward scrutiny but can never rescue a failed gate, and an advisory**mutation smoke**whose survivors are never revealed to the fixing agent — the Goodhart guard. One click reverts everything an episode touched, untracked files included. Two ledgers, both local, neither uploaded. **Context-pack composition**is a factored`2²`arm grid (graph-slice depth × runtime-evidence inclusion). Selection is Thompson sampling with an ε-floor mixture, and the**exact mixture propensity is logged with each decision**— the requirement for unbiased IPS/SNIPS/DR later. The ε-floor decays but never reaches zero, so importance weights stay bounded. Posteriors count**only objective episodes**— a user abort or an "approve as done" click is not evidence about arm quality. Attribution is a**COMA-style counterfactual**per feature over the grid of posterior means:*"did runtime evidence help"*is a computed number, not a claim. **Retrieval serving**is a separate ε-greedy bandit (over the top-k action set) with its own ledger. A candidate config is promotable only when*all*hold: ESS ≥ 25, n ≥ 40 joined samples, ≥ 8 logged pulls per compared action, BCa-bootstrap 95% lower bound of the doubly-robust delta ≥ 0, zero clipped weights, and no epoch contradicting the pooled delta (the Simpson-artifact guard). A promoted policy then serves at**5% canary only**, and three consecutive negative canary rewards roll it back automatically. Measured on this repo's own ledger (800 logged decisions, 770 joined, 12 index epochs —](https://github.com/huggingface/smolagents/pull/2572)[`docs/learning.md`): **The gate admitted exactly the measured winner and blocked both the uncertain and the unsupported candidate.**That is what "closed-loop" is being asked to mean here — and the parts that are*not*online learning (the referee's thresholds, the oracle catalogue, the fault shapes) are fixed policy, deliberately. Give your coding agent runtime context — one loop, these capabilities: - **Semantic code search**— ask by meaning, get ranked symbols with`def`bodies and line numbers, embedded by a local model (no cloud keys). - **Code Graph**— a persistent map of every symbol and call edge, updated incrementally on save, with a live runtime overlay. - **Runtime tracing**— zero-edit runtime tracing for AI coding agents: timing, memory, args, returns, errors — per call, joined to source. - **Rank suspects**— on any failure, symbols ranked by fault-localization score over real pass/fail requests, error messages attached. - **Verified fixes**— verify AI-generated code actually works: replayed start, live port, acceptance tests the agent never sees. One click reverts everything an episode touched. - **Dead code sections**—*"View Dead Code"*explains every untraced island. Sections split into**no references**and**reached from live code but never taken**, each with the callers that still point at it and a keep-or-cut verdict with its reasoning. - **Recoverable time**— latency hotspots ranked by the milliseconds you would actually get back, each dispatched as a predicted-then-proven optimization instead of a guess about what is slow. - **Ask Vinv**— ask anything about your running system in plain English; every answer cites the exact trace spans and source lines it came from, and a**deterministic critic**blocks any claim the evidence can't back — grounded Q&A, not confident guessing. - **Behavior exerciser**— Vinv doesn't wait for traffic: it drives**every discovered service endpoint itself**, picks strategies with a Thompson-sampling bandit rewarded by oracle violations first and new coverage only as a bonus, and turns every response into a permanent regression case. - **Journey**— one walkthrough of everything verified: every service, then every endpoint's call tree, latency flamegraph, and the exact inputs → outputs exercised — with a form to add your own test inputs that the engine replays forever after. - **Auto-Pilot & the red ring**— one click drives discover → set up → trace → exercise → fix → verify until green or budget; when new trace errors land, the fix episode is*already dispatched*by the time you see the red ring in the graph. The budget is yours: set attempts per service in**Configure**, and when a run exhausts them Vinv asks whether to grant more instead of quietly giving up. - **Agent babysitting**— a doom-loop guard (token-set self-similarity) catches a repeating agent, an adaptive silence watchdog catches a hung one, and**"Dispute a Verified Fix"**keeps even the verifier accountable. - **Findings**— what Vinv found and what it fixed, with the statistical evidence: issue clusters, optimization episodes with paired-bootstrap confidence intervals, regression diff kinds, and a machine-readable`findings.json`your agent can consume directly. **Honest scope:**Python first —**services and APIs**. Auto-Pilot discovers runnable services, brings them up under tracelens, and exercises their HTTP (and related) entrypoints. Other languages get the index, graph and grounded QnA, but no runtime evidence yet. TypeScript and Go next. ## Why agents don't reward-hack under Vinv Vinv ties**every runtime trace to the exact code segment that produced it**and hands your agent a context graph built from that join — so the agent argues from evidence, not vibes. And when the agent claims victory, Vinv doesn't take its word: - **Acceptance tests are authored*before*the fix**, stored outside the workspace under an opaque token, and must fail deterministically twice on the broken code — it can't train to the test, and a test that passes pre-fix is thrown away. - **A "faster" fix that changes any observable output is auto-reverted**— the behavior suite must replay byte-identical, and the speedup's paired-bootstrap 95% CI must exclude zero. Faster-but-wrong never lands. - **Deliberate 4xx rejections aren't "errors" to fix**— the defect classifier knows the difference between a service saying*no*correctly and a service breaking, so the agent is never handed a fake goal it can only game. Same rule in the newer oracles: a sandbox that refuses`nonlocal`is enforcing a documented limit, not failing. - **Silent-wrong-value findings dispatch with value-shaped criteria**— for`differential-mismatch`,`fault-divergence`,`concurrency-divergence`,`invariant-violation`and`baseline-degraded`, "these calls no longer raise" would be vacuous, because the target never raised. The criterion is the*value*. - **When two attempts stop making progress, a Nash-bargaining stall judge decides**— continue only if both an explorer stance*and*an auditor stance strictly prefer it to asking you. Otherwise you get a judgment panel, not a token bonfire. Traffic only shows you the code paths users happen to hit. The behavior exerciser drives the rest — same](https://github.com/VinvAI/VinvAI/blob/HEAD/docs/learning.md)[fastapi/full-stack-fastapi-template, one run: The**authenticated sweep**(every endpoint replayed under credentials the login scenario captured, with freshly created resource IDs fed to the by-id endpoints) surfaced**four real bugs**that anonymous traffic can never reach: - `GET /api/v1/users/`→**HTTP 500**— an invalid email stored by an unvalidated private endpoint poisons response serialization - `POST /api/v1/private/users/`→**IntegrityError escapes as a 500**—`email: str`instead of`EmailStr`, no duplicate guard - `POST /api/v1/utils/test-email/`→**HTTP 500**—`assert settings.emails_enabled`crashes instead of degrading - `POST /api/v1/password-recovery-html-content/{email}`→**connection killed**— unsanitized header rendering The harness then**fixed all four**, and the regression suite now distinguishes*your code regressed*from*the test engine's own leftover data changed the world*(the state ledger) — so a re-run doesn't cry wolf. Phantom perf regressions are gone too: a latency diff must survive a median of 5 replays before it's reported. Vinv is an MCP server for Claude Code and Cursor — and every other MCP client you already use. One command (**Register Vinv MCP in Agent Tools**) writes the servers into every agent it detects: Registration is idempotent and never commits secrets. The servers (`vinv-index`,`vinv-runtime`,`vinv-exercise`) launch over stdio via the editor's own runtime. - **Claude Code**—`~/.claude.json`, project-local scope (no trust prompt). Verify:`claude mcp list`shows the Vinv servers. - **Cursor**—`<repo>/.cursor/mcp.json`. Verify: Settings → MCP shows them green. - **Codex CLI**—`~/.codex/config.toml`under`](https://github.com/fastapi/full-stack-fastapi-template)[mcp_servers.vinv-index]`/`[mcp_servers.vinv-runtime]`. - **Copilot Chat**— native VS Code MCP provider (auto),`.vscode/mcp.json`on older builds. - **Windsurf Cascade**—`~/.codeium/windsurf/mcp_config.json`. - **Gemini CLI**— dispatch works out of the box; for MCP tools, add the same stdio servers to`~/.gemini/settings.json`. **Your agent is also Vinv's only LLM**— every analysis step routes through the coding-agent CLI you already pay for. No provider keys, no model picker. Vinv's release gate is Vinv — these numbers come from running the loop on this repository: ``` `flowchart LR T[Trace] --> I[Index] --> S[Serve MCP] --> V[Verify] --> L[Learn] --> T` ``` - **Trace**— run your Python service under the bundled tracer: no SDK, no code changes. (No service? The function, differential, fault, concurrency and environment oracles need none.) - **Index**— every function embedded locally into a semantic index + call graph. - **Serve**— MCP servers hand the evidence to your agent. - **Verify**— replayed start, live port, acceptance tests generated*before*the fix. - **Learn**— propensity-logged decisions; retrieval and pack composition update only on off-policy-evaluation wins, behind a 5% canary with automatic rollback. No black boxes — every decision Vinv makes has a published method behind it, and each one exists to keep the loop honest, not clever: The full learning walk — reward, propensity, gating math, with`file:line`for every claim — is[`docs/learning.md`. The test ontology is](https://github.com/VinvAI/VinvAI/blob/HEAD/docs/learning.md)[`docs/testing-ontology.md`. Vinv indexes**the code**and generates — from your own run —**the traces**and**the metrics derived from them**, then ties all three to the exact function that handled each request. The artefacts are commodities;**the join is not.**Auto-Pilot drives the whole loop unaided: discover services → set up via your agent → start under tracing → exercise → fix → re-verify, until green or budget. Layout:](https://github.com/VinvAI/VinvAI/blob/HEAD/docs/testing-ontology.md)[`extension/`(editor UI + MCP servers),](https://github.com/VinvAI/VinvAI/blob/HEAD/extension/)[`index/`(Rust semantic index),](https://github.com/VinvAI/VinvAI/blob/HEAD/index/)[`embedder/`(local](https://github.com/VinvAI/VinvAI/blob/HEAD/embedder/)[CodeRankEmbedsidecar),](https://huggingface.co/nomic-ai/CodeRankEmbed)[`tracelens/`(zero-edit tracer),](https://github.com/VinvAI/VinvAI/blob/HEAD/tracelens/)[`exerciser/`(the oracle swarm + campaign bandit),](https://github.com/VinvAI/VinvAI/blob/HEAD/exerciser/)[`identification/`(trace↔source join),](https://github.com/VinvAI/VinvAI/blob/HEAD/identification/)[`handbook/`·](https://github.com/VinvAI/VinvAI/blob/HEAD/handbook/)[`bringup/`·](https://github.com/VinvAI/VinvAI/blob/HEAD/bringup/)[`goal/`(discovery & episodes),](https://github.com/VinvAI/VinvAI/blob/HEAD/goal/)[`tests/e2e/`(planted-bug golden test). Python engines are one](https://github.com/VinvAI/VinvAI/blob/HEAD/tests/e2e/)[uvworkspace. - **Hunt without a server**—`exerciser campaign <repo> --budget 20`. No service, no`--base-url`: the function, differential, fault, concurrency and environment oracles do the work, and the bandit reports which technique paid. - **Exercise your API**—`exerciser plan <repo> && exerciser run <repo> --base-url http://127.0.0.1:PORT`(or let Auto-Pilot's`exercise`phase do it). An**environment canary**first dry-runs your login chains and tells you*loudly*if the database was reset or credentials unseeded — no more silently-401 runs. - **Find your dead code**— Command Palette →**"Vinv: View Dead Code"**: every untraced section, split into never-referenced vs reachable-but-never-taken, with the live callers that still point at it and a keep-or-cut verdict. - **Walk everything**—**"Vinv: Open Journey"**. Overview first (services, coverage, open issues), then`Next`/`→`through every endpoint: call tree with live runtime, flamegraph, and the exact inputs → outputs driven. Hover anything cryptic — every marker explains itself. - **Add your own test input**— on any Journey endpoint step, fill body/params/expected status and hit*Add input*. It lands in the same plan layer the AI-authored scenarios use, runs with the endpoint's auth setup on the next exercise, and becomes a permanent regression case. - **See what got fixed**—**"Vinv: Open Findings"**: issue clusters, optimization episodes with their confidence intervals, regression diffs by kind, latency profile, cleanup ledger. The backing file`.vinv/reports/findings.json`is the same data, machine-readable — point your agent at it. - **Regress after any change**—`exerciser regress <repo> --base-url …`replays all banked cases (re-capturing fresh credentials itself) and reports**behavior / contract / perf / environment**diffs separately, so environment drift never masquerades as a code regression. - **Hunt waste on demand**—**"Optimize Latency Hotspots"**,**"Analyze Memory Trends"**(Theil–Sen leak suspects), and**"Analyze Cache Opportunities"**each turn one command into an evidence-seeded fix episode — accepted only if the paired-bootstrap CI clears and behavior stays byte-identical. `pip install vinv`ships every engine as a console script — or use any one standalone: Requires`identification consolidate`first for`apis.json`, and — for real coverage — a service running under`tracelens`. **`vinv-index`**— your services' code and the session: **`vinv-runtime`**— the captured runs (read-only, provenance-stamped): **`vinv-exercise`**— closes the endpoint-testing loop: your agent exercises your service and reports the run back, and Vinv grades what came back. - **Everything on your machine**— per-repo state in`.vinv/`(auto-gitignored), per-machine in`~/.vinv/`. No account, no API keys,**no telemetry — none.** - **One outbound request, and you can read it**: a GET of a static JSON file at`notices.vinv.ai`on activation, for broken-release and security notices only. No query string, no identifiers, no version, nothing uploaded; at most once per 12 hours; the URL is a constant no setting can repoint; disable with`vinv.notices.enabled`. - The only download is the embedding model (Hugging Face, once, ~500 MB); everything else builds from this repo. - Traces store bounded**summaries**, not raw values; sensitive parameter names (`password`,`token`,`api_key`, …) are redacted, never captured. - The only LLM Vinv talks to is the coding-agent CLI**you**configured, through its own auth. See](https://docs.astral.sh/uv/)[CONTRIBUTING.md—`uv sync`,`cargo build`in`index/`,`npm install && npm run check`in`extension/`, keep`tests/e2e/planted_bug_golden/run.py`green. Good first issues are labeled. By taking part you agree to our](https://github.com/VinvAI/VinvAI/blob/HEAD/CONTRIBUTING.md)[Code of Conduct; to report a vulnerability, see](https://github.com/VinvAI/VinvAI/blob/HEAD/CODE_OF_CONDUCT.md)[SECURITY.md.](https://github.com/VinvAI/VinvAI/blob/HEAD/SECURITY.md)[Apache License 2.0© 2026 VinvAI. **Your agent says it's done. Vinv says prove it.** If Vinv caught something your agent missed —](https://github.com/VinvAI/VinvAI/blob/HEAD/LICENSE)[leave a review on Open VSXand ⭐ star this repo. ](https://open-vsx.org/extension/VinvAI/VinvAI/reviews)[vinv.ai·](https://vinv.ai)[Open VSX·](https://open-vsx.org/extension/VinvAI/VinvAI)[LinkedIn·](https://www.linkedin.com/company/vinvai/)[](https://github.com/VinvAI/VinvAI/blob/HEAD/mailto:support@vinv.ai)<<<[support@vinv.ai· Python first, TS & Go next ·**Context beats model size.**](https://github.com/VinvAI/VinvAI/blob/HEAD/mailto:support@vinv.ai)