Scrapling Fetch

by cyberchitta

16 stars
5.5k downloads
Not rated
GitHub

About

Enables AI to access text content from websites protected by bot detection mechanisms through three protection levels (basic, stealth, max-stealth), retrieving complete pages or specific content patterns without manual copying.

Details

Author
cyberchitta
Repository
cyberchitta/scrapling-fetch-mcp
GitHub stars
16
Downloads
5,450
License
Apache License 2.0
Categories
Web Scraping, Automation, Other, Developer Tools, Design, Workplace, File Management, AI, Knowledge Base, Frontend, Infrastructure
Tags
#web

- Page fetching with pagination support
- Pattern extraction using regular expressions
- Three automatic protection levels: basic, stealth, max-stealth
- Text-only content (HTML / Markdown output)
- Optimized for low-volume retrieval of documentation

Setting up with Highlight

This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:

  1. Download and install Highlight from highlightai.com/download
  2. Navigate to the plugins tab and select "Add Custom Plugin"
  3. Configure the plugin with the settings below
    Plugin Name Scrapling Fetch
    Command (node, npx, python, etc.) uvx
    Arguments
    • Argument 1 scrapling-fetch-mcp

    Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.

  4. Enable "Start Automatically" if you want the plugin to start when Highlight launches

From the repository

The easiest way is as a Claude Code skill: clone the repository, copy the skills/s-fetch folder into your project’s .claude/skills/ (or ~/.claude/skills/ for all projects), then invoke /s-fetch. For Claude Desktop, install the tool with uv tool install git+https://github.com/cyberchitta/scrapling-fetch-mcp, then add the scrapling-fetch MCP server configuration to your claude_desktop_config.json.

Claude Desktop / Cursor

Paste into your MCP client config file to install this server.

{
    "mcpServers": {
        "scrapling fetch": {
            "cwd": "optional",
            "env": {},
            "args": [
                "scrapling-fetch-mcp"
            ],
            "shell": false,
            "command": "uvx"
        }
    }
}

Linux

{
    "cwd": "optional",
    "env": [],
    "args": [
        "scrapling-fetch-mcp"
    ],
    "shell": false,
    "command": "uvx"
}

Macos

{
    "cwd": "optional",
    "env": [],
    "args": [
        "scrapling-fetch-mcp"
    ],
    "shell": false,
    "command": "uvx"
}

Windows

{
    "cwd": "optional",
    "env": [],
    "args": [
        "scrapling-fetch-mcp"
    ],
    "shell": false,
    "command": "uvx"
}

Fetches HTML and markdown from websites with anti-automation measures using Scrapling.

Helps AI assistants fetch content from bot-protected websites. Uses Scrapling (patchright + curl-cffi) to bypass anti-automation measures, returning clean HTML or Markdown.

Optimized for low-volume retrieval of documentation and reference materials. Not designed for high-volume scraping or data harvesting.

The easiest way to use this is as a Claude Code skill. Once installed, Claude will automatically fetch bot-protected URLs when you ask — no manual commands needed.

Install into your project(recommended — only loads in this project's context):

git clone --depth=1 https://github.com/cyberchitta/scrapling-fetch-mcp /tmp/scrapling-fetch-mcp cp -r /tmp/scrapling-fetch-mcp/skills/s-fetch .claude/skills/ rm -rf /tmp/scrapling-fetch-mcp

Or install for all projects(loads into context everywhere):

git clone --depth=1 https://github.com/cyberchitta/scrapling-fetch-mcp /tmp/scrapling-fetch-mcp cp -r /tmp/scrapling-fetch-mcp/skills/s-fetch ~/.claude/skills/ rm -rf /tmp/scrapling-fetch-mcp

The first time you invoke/s-fetch, Claude will readskills/s-fetch/references/install.mdand run the one-time setup (installs the tool and downloads browser binaries). After that, just ask naturally:

"Fetch the docs at https://example.com/api" "Find all mentions of 'authentication' on that page" "Get me the installation instructions from their homepage"

If the/s-fetchskill has already installed the tool, skip to the config below.

uv tool install git+https://github.com/cyberchitta/scrapling-fetch-mcp uvx --from git+https://github.com/cyberchitta/scrapling-fetch-mcp scrapling install

Note: Browser installation downloads hundreds of MB and must complete before first use. If the server times out initially, wait a few minutes and try again.

Add this to your Claude Desktop MCP settings and restart:

MacOS:~/Library/Application Support/Claude/claude_desktop_config.json
Windows:%APPDATA%\Claude\claude_desktop_config.json

{ "mcpServers": { "scrapling-fetch": { "command": "uvx", "args": ["scrapling-fetch-mcp"] } } }

Two tools, used automatically by Claude:

- Page fetching— retrieves complete pages with pagination support
- Pattern extraction— finds content matching a regex

Three protection levels, escalated automatically:

- basic— fast (1-2s), works for most sites
- stealth— moderate (3-8s), headless Chromium
- max-stealth— thorough (10s+), full browser fingerprint

- Text content only (documentation, articles, references)
- Not for high-volume scraping or sites requiring authentication
- Performance varies by site complexity and protection level

A server for web crawling and content extraction using the Crawl4AI library.

Scrape any URL with JavaScript rendering and get back clean markdown — built for AI agents, LLM pipelines, and autonomous research workflows.

Fetch and extract web content using a Playwright headless browser, with support for intelligent extraction and flexible output.

Ultra-fast web scraper and deep discussion crawler delivering clean Markdown for AI Agents

A web crawler and content extractor that supports multiple output formats like text, markdown, and JSON.

Fetches and transforms web content, including JavaScript-rendered pages and media files, into various formats.

A Python-based MCP server for headless web scraping. It extracts the main text content from web pages and outputs it as Markdown, text, or HTML.

Website change monitoring with URL watching, snapshot comparison, and intelligent content extraction for AI assistants

Quick Start: npx -y @aimino/fast-html-mcp-server

MCP Server for Xiaohongshu (小红书) — let AI search notes, read content, and publish via browser automation

No reviews yet — be the first

Sign in to leave a review

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.