Open Crawler MCP Server

by naoto24kawa

Not rated
GitHub

About

A web crawler and text extractor with robots.txt compliance, rate limiting, and page size protection.

Details

Author
naoto24kawa
Categories
Web Scraping, Automation, Other

Setup

Install Open Crawler MCP Server in your MCP client (Claude Desktop, Cursor, Windsurf, and others).

Repository: https://github.com/naoto24kawa/open-crawler-mcp-server

Follow the installation instructions in the repository README, then restart your MCP client.

A web crawler and text extractor with robots.txt compliance, rate limiting, and page size protection.

Enable AI agents to get structured data from unstructured web with AgentQL.

Easy web data access. Simplified retrieval of information from websites and online sources.

Turn websites into datasets with Scrapezy

Scrapfly MCP Server gives AI agents a simple, unified way to scrape live web data with built-in anti-bot handling.

Interact with WebScraping.AI for web data extraction and scraping.

Retrieve structured data through natural language conversations

CrawlForge MCP is a production-ready MCP server with 18 web scraping tools for AI agents. It gives Claude, Cursor, and any MCP-compatible client the ability to fetch URLs, extract structured data with CSS/XPath selectors, run deep multi-step research, bypass anti-bot detection with TLS fingerprint randomization, process documents, monitor page changes, and more. Credit-based pricing with a free tier (1,000 credits/month, no credit card required).

A high-performance x402 intelligence gateway providing 20+ social and web endpoints. Powered by 6-stage routing logic—including cloud-rendered JS fetches, residential IP rotation, and automatic settlement refunds for blocked hosts. Built for AI Agents that require reliable, pay-per-request access to LinkedIn, Reddit, Instagram, and beyond without API key management

Scrape, crawl, and extract data from any website using the Firecrawl API.

No reviews yet — be the first

Sign in to leave a review

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.