GitPrism
About
GitPrism is a fast, token-efficient, stateless pipeline that converts public GitHub repositories into LLM-ready Markdown.
Details
- Author
- cougz
- Categories
- Web Scraping, Other, Developer Tools
Jump to
Setup
Install GitPrism in your MCP client (Claude Desktop, Cursor, Windsurf, and others).
Repository: https://github.com/cougz/GitPrism
Follow the installation instructions in the repository README, then restart your MCP client.
A fast, token-efficient, stateless pipeline that converts public GitHub repositories into LLM-ready Markdown. Deployed as a single Cloudflare Worker serving humans, AI agents, and MCP clients from one shared core engine.
┌─────────────────────────────────────────────┐ │ Single Cloudflare Worker │ │ (gitprism) │ │ │ Humans ────────► │ / → Astro Static UI │ │ (Workers Static Assets) │ │ │ AI Agents ─────► │ /ingest?... → REST API │ │ /<github-url> → URL Proxy (shorthand) │ │ │ MCP Clients ───► │ /mcp → Stateless MCP Server │ │ (createMcpHandler) │ │ │ │ ┌───────────────────┐ │ │ │ Core Engine │ │ │ │ URL Parser │ │ │ │ Zipball Fetch │ │ │ │ fflate Decomp │ │ │ │ Filter/Ignore │ │ │ │ MD Formatter │ │ │ └───────────────────┘ │ └─────────────────────────────────────────────┘ │ ▼ GitHub Zipball API (authenticated via secret)
GitPrism supports optional GitHub personal access tokens to bypass shared rate limits and use your personal GitHub API quota.
- Without authentication:30 requests per minute per IP address
- With server token:Shared across all users (5,000 req/hr for the Worker's IP)
- With user token:Your personal GitHub quota (5,000 requests per hour)
curl -H "X-GitHub-Token: ghp_xxxx" https://gitprism.cloudemo.org/ingest?repo=owner/repo
Web UI:Click the ⚙️ settings icon in the search bar to open the token settings panel. Your token is stored inlocalStorageand never sent to any server other than GitHub via this Worker.
{ "url": "https://github.com/owner/repo", "detail": "full", "github_token": "ghp_xxxx" }
- Scope:Contents: Read-only
- Repository access:Public repositories only
- Create token:https://github.com/settings/tokens?type=beta
TheX-Token-Sourceheader indicates which token was used:
- user— Your personal token (viaX-GitHub-Tokenheader)
- server— The Worker's shared token
- none— No token available
Visithttps://gitprism.cloudemo.org/and paste any GitHub URL. Use the settings panel (⚙️) to configure your GitHub token.
Canonical form (recommended for programmatic use):
GET /ingest?repo=owner/repo&ref=main&path=src&detail=full
URL-appended shorthand (human-friendly):
GET /https://github.com/owner/repo/tree/main/src
Branch, ref, and subdirectory are automatically extracted from the GitHub URL. Append a detail shorthand to control output:
GET /https://github.com/owner/repo?summary GET /https://github.com/owner/repo/tree/main/src?file-list
Detail level shorthand— instead of?detail=<level>, append the level as a bare key. Works on both the canonical and URL-proxy forms:
/ingest?repo=owner/repo&summary /https://github.com/owner/repo?structure
Connect any MCP-compatible client tohttps://gitprism.cloudemo.org/mcp.
{ "url": "https://github.com/owner/repo", "detail": "summary", "github_token": "ghp_xxxx" }
The tool is fully compatible with Code Mode agents — the strongly-typed Zod input schema and descriptive annotations allow client-sidecreateCodeTool()to wrap it automatically.
Workers Builds connects your GitHub repo to Cloudflare and deploys automatically on every push tomain. The Astro UI is compiled during the build step;ui/dist/is intentionally not committed to git.
-
Go to theCloudflare dashboard→Workers & Pages→Create→Import a Git repository
Connect your GitHub account and select this repo
ClickSave and Deploy— the first build will run immediately
Once deployed, go to your Worker →Settings→Variables and Secrets→Adda secret:
Without this secret the Worker still functions, but GitHub API rate limits drop from 5,000 to 60 requests/hour (shared across all requests from the Worker's outbound IP).
Optional — Custom domain:Worker →Settings→Custom Domains→ add your domain. This enables the Workers Cache API. Without a custom domain the Worker deploys to<name>.<subdomain>.workers.devand caching silently no-ops (the code handles this gracefully). To enable routing once you have a domain, uncomment and update theroutesblock inwrangler.jsonc:
"routes": [ { "pattern": "yourdomain.com/", "custom_domain": true } ],
git clone https://github.com/cougz/gitprism.git cd gitprism npm install npm run build # builds ui/dist/ npx wrangler secret put GITHUB_TOKEN npx wrangler deploy
Configured inwrangler.jsoncundervars. Override in the Cloudflare dashboard under Worker →Settings→Variables and Secretsif needed:
ui/dist/(the compiled Astro frontend) is excluded from git. Wrangler readsassets.directory = "./ui/dist"fromwrangler.jsoncand uploads those files as static assets during deploy. If that directory does not exist at deploy time, the Worker deploys with no UI. Thenpm run buildstep compiles the Astro source inui/src/intoui/dist/before Wrangler runs.
# Build the Astro UI (required before deploying or running wrangler dev) npm run build # Run tests (169 tests) npm test # Watch mode npm run test:watch # Type-check npm run typecheck # Local dev server (requires ui/dist/ to exist — run npm run build first) npm run dev
gitprism/ ├── src/ │ ├── index.ts # Worker entry point, routing │ ├── types.ts # Shared interfaces and error classes │ ├── engine/ │ │ ├── parser.ts # URL parsing and validation │ │ ├── fetcher.ts # GitHub zipball download + size check │ │ ├── decompressor.ts # fflate decompression + processing │ │ ├── filter.ts # Ignore lists, .gitignore, binary detection │ │ ├── formatter.ts # Markdown output generators (4 levels) │ │ └── ingest.ts # Shared pipeline (used by API + MCP) │ ├── mcp/ │ │ └── server.ts # createMcpHandler setup │ ├── api/ │ │ ├── handler.ts # REST API handler, streaming, caching │ │ └── llmstxt.ts # /llms.txt endpoint │ └── utils/ │ ├── cache.ts # Workers Cache API helpers │ ├── ratelimit.ts # Rate limiting helper │ └── headers.ts # Response header builder ├── test/ # Vitest test files (169 tests) ├── ui/ │ ├── src/ # Astro source │ ├── dist/ # Build output (gitignored) │ └── astro.config.mjs ├── PLAN.md # Detailed implementation plan └── wrangler.jsonc
The following are always excluded regardless of.gitignore:
Directories:node_modules/,vendor/,.git/,__pycache__/,.venv/,venv/,dist/,build/,.next/,.nuxt/,.svelte-kit/,.output/,.cache/,.parcel-cache/,coverage/,.tox/,.mypy_cache/
Files:package-lock.json,yarn.lock,pnpm-lock.yaml,bun.lockb,Cargo.lock,composer.lock,Gemfile.lock,go.sum,poetry.lock,.min.js,.min.css,.map,.wasm,.pb.go,.pyc,.pyo
Binary extensions:.png,.jpg,.jpeg,.gif,.ico,.webp,.bmp,.tiff,.svg,.woff,.woff2,.ttf,.eot,.otf,.pdf,.zip,.tar,.gz,.bz2,.7z,.rar,.exe,.dll,.so,.dylib,.bin,.o,.a,.mp3,.mp4,.avi,.mov,.mkv,.flac,.wav,.ogg,.sqlite,.db,.DS_Store
Binary content detection:Files containing null bytes in their first 8 KB are skipped regardless of extension.
The root.gitignoreof the repository is parsed and applied. Supports:
- Wildcard patterns (.log,/.tmp)
- Directory patterns with trailing slash (logs/)
- Rooted patterns (/build)
- Negation patterns (!important.log)
- Comments (# this line is ignored)
Limitation:Only the root.gitignoreis evaluated. Nested.gitignorefiles (e.g.,src/.gitignore) are not supported in v1.
Theingest_repoMCP tool is compatible with Code Mode agents by design:
- Clear, descriptive tool name (ingest_repo)
- Multi-sentence description explaining all four detail levels
- Strongly-typed Zod schemas with.describe()on every parameter
- No server-side changes needed — standard MCP tools with typed schemas are inherently Code Mode compatible
- WithoutX-GitHub-Token:Cloudflare rate limiter enforces 30 requests per minute per IP
- WithX-GitHub-Token:**Cloudflare rate limiter is bypassed; your personal GitHub quota applies (5,000 req/hr)
- TheX-Token-Sourceresponse header indicates which token was used (user,server, ornone)
Cache keys use resolved commit SHAs for automatic invalidation when repos update. Old cache entries expire naturally after TTL. If SHA resolution fails, caching is skipped and fresh data is always fetched.
Convert GitHub repositories to MCP servers automatically. Extract tools from OpenAPI, GraphQL & REST APIs for Claude Desktop, Cursor, Windsurf, Cline & VS Code. AI-powered code generation creates type-safe TypeScript/Python MCP servers. Zero config setup - just paste a repo URL. Built for AI assistants & LLM tool integration.
Official GitHub MCP server for repository search, issues, pull requests, code context, and GitHub workflows in MCP-compatible AI clients.
Six read-only decision tools for coding agents to assess GitHub bounties, audit agent instructions, diagnose GitHub Actions failures and flakes, and detect MCP tool drift. Remote Streamable HTTP
Integrate with CodeRabbit AI for automated code reviews, pull request analysis, and report generation.
Integrates Google's Gemini for advanced codebase analysis, web search, and processing of text, PDFs, and images.
A server for managing and interacting with GitHub Gists, built with Go.
Analyze and query GitHub repositories using the GitHub Chat API.
A production-ready MCP server that connects any MCP-compatible AI agent to the GitHub API. Manage repositories, issues, pull requests, and search — all through natural language.
Fetches trending repositories from GitHub and saves them as Markdown files. Supports proxy configuration.
A Grey Hack server for Cursor IDE, providing GitHub code search, Greybel-JS transpilation, API validation, and script generation.
Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.




