ScrAPI MCP Server

by deventerprisesoftware

3 stars
449 downloads
Not rated
GitHub Website

About

Enables web scraping from sites with bot detection, captchas, or geolocation restrictions through residential proxies and automated captcha solving for content extraction in HTML or Markdown formats.

Details

Author
deventerprisesoftware
Repository
DevEnterpriseSoftware/scrapi-mcp
GitHub stars
3
Downloads
449
License
MIT License
Categories
Web Scraping, API, Other, Productivity, AI, Developer Tools, Search, Infrastructure, Knowledge Base, Project Management

Jump to

- Scrape any valid https:// or http:// URL through ScrAPI.
- Return either raw HTML or readable Markdown.
- Run browser commands before scraping.
- Use stdio transport for desktop MCP clients.
- Use Streamable HTTP transport for remote MCP clients and local testing.
- Run with npx, Docker, Smithery, or from source.

Setting up with Highlight

This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:

  1. Download and install Highlight from highlightai.com/download
  2. Navigate to the plugins tab and select "Add Custom Plugin"
  3. Configure the plugin with the settings below
    Plugin Name ScrAPI MCP Server
    Command (node, npx, python, etc.) npx
    Arguments
    • Argument 1 -y
    • Argument 2 @deventerprisesoftware/scrapi-mcp
    Environment
    • SCRAPI_API_KEY your-scrapi-api-key

    Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.

  4. Enable "Start Automatically" if you want the plugin to start when Highlight launches

From the repository

Install via npx, Docker, or Smithery, and set the SCRAPI_API_KEY environment variable. Use the provided tools scrape_url_html and scrape_url_markdown with a target URL and optional browser commands. Configure your MCP client (e.g., Claude Desktop, Cursor, VS Code) with stdio or Streamable HTTP transport.

scrape_url_html

Scrapes a URL and returns the result as HTML. Parameters: url (string, required), browserCommands (string, optional - JSON array string of browser commands to execute before scraping).

scrape_url_markdown

Scrapes a URL and returns the result as Markdown. Parameters: url (string, required), browserCommands (string, optional - JSON array string of browser commands to execute before scraping).

Claude Desktop / Cursor

Paste into your MCP client config file to install this server.

{
    "mcpServers": {
        "scrapi mcp server": {
            "env": {
                "SCRAPI_API_KEY": "your-scrapi-api-key"
            },
            "args": [
                "-y",
                "@deventerprisesoftware/scrapi-mcp"
            ],
            "command": "npx"
        }
    }
}

Linux

{
    "env": {
        "SCRAPI_API_KEY": "your-scrapi-api-key"
    },
    "args": [
        "-y",
        "@deventerprisesoftware/scrapi-mcp"
    ],
    "command": "npx"
}

Macos

{
    "env": {
        "SCRAPI_API_KEY": "your-scrapi-api-key"
    },
    "args": [
        "-y",
        "@deventerprisesoftware/scrapi-mcp"
    ],
    "command": "npx"
}

Windows

{
    "env": {
        "SCRAPI_API_KEY": "your-scrapi-api-key"
    },
    "args": [
        "/c",
        "npx",
        "-y",
        "@deventerprisesoftware/scrapi-mcp"
    ],
    "command": "cmd"
}

A server for scraping web pages using the ScrAPI API.

ScrAPI MCP Server lets MCP-compatible clients scrape web pages throughScrAPI.

- Features
- Available tools
- Prerequisites
- API key
- Quick start
- MCP client setup
- HTTP transport
- Cloud-hosted server
- Usage examples
- Browser commands
- Troubleshooting
- Development
- License

- Scrape any validhttps://orhttp://URL through ScrAPI.
- Return either raw HTML or readable Markdown.
- Run browser commands before scraping.
- Use stdio transport for desktop MCP clients.
- Use Streamable HTTP transport for remote MCP clients and local testing.
- Run withnpx, Docker, Smithery, or from source.

Scrapes a URL and returns the result as HTML.

Use this when you need the page structure, links, tables, embedded metadata, or custom downstream parsing.

- text/htmlcontent from the requested page.
- isError: truewith the ScrAPI error body when the upstream request fails.

Scrapes a URL and returns the result as Markdown.

Use this when the text content matters more than the HTML structure, for example article extraction, product copy, search result summaries, or LLM-friendly page analysis.

- text/markdowncontent from the requested page.
- isError: truewith the ScrAPI error body when the upstream request fails.

Choose one of the following runtime options:

- Node.js 18 or newer fornpxor local development.
- Docker for container-based usage.
- An MCP-compatible client such as Claude Desktop, MCP Inspector, or another client that supports stdio or Streamable HTTP MCP servers.

SetSCRAPI_API_KEYto use your ScrAPI account:

export SCRAPI_API_KEY="your-scrapi-api-key"

$env:SCRAPI_API_KEY = "your-scrapi-api-key"

An API key is required. Without one, ScrAPI currently allows limited free usage on certain domain with lower concurrency and queueing priority.

The default transport is stdio, which is the transport most desktop MCP clients use when they launch a local server process.

npx -y @deventerprisesoftware/scrapi-mcp

SCRAPI_API_KEY="your-scrapi-api-key" npx -y @deventerprisesoftware/scrapi-mcp

$env:SCRAPI_API_KEY = "your-scrapi-api-key" npx -y @deventerprisesoftware/scrapi-mcp

The published Docker image starts in HTTP mode by default and listens on port5000.

docker run --rm -p 5000:5000 -e SCRAPI_API_KEY="your-scrapi-api-key" deventerprisesoftware/scrapi-mcp

To run the container as a stdio server for a local MCP client:

docker run -i --rm -e TRANSPORT=stdio -e SCRAPI_API_KEY="your-scrapi-api-key" deventerprisesoftware/scrapi-mcp

Most local coding assistants use one of these two configuration shapes:

- Stdio: the client starts this package withnpxor Docker and communicates over stdin/stdout.
- Streamable HTTP: you start this server yourself withTRANSPORT=http, then point the client athttp://localhost:5000/mcpor your deployed URL.

When a client has a tool timeout setting, use a value close to300000milliseconds or300seconds. ScrAPI can take several minutes for pages that require CAPTCHA solving, browser rendering, or multiple browser commands.

Add this to yourclaude_desktop_config.json:

{ "mcpServers": { "ScrAPI": { "command": "npx", "args": ["-y", "@deventerprisesoftware/scrapi-mcp"], "env": { "SCRAPI_API_KEY": "your-scrapi-api-key" } } } }

{ "mcpServers": { "ScrAPI": { "command": "docker", "args": [ "run", "-i", "--rm", "-e", "SCRAPI_API_KEY", "deventerprisesoftware/scrapi-mcp" ], "env": { "SCRAPI_API_KEY": "your-scrapi-api-key" } } } }

After changing the config, restart Claude Desktop. You should see the two ScrAPI tools available in the MCP tools list.

Cursor supports project configuration at.cursor/mcp.jsonand global configuration at~/.cursor/mcp.json. See theCursor MCP documentation.

{ "mcpServers": { "scrapi": { "command": "npx", "args": ["-y", "@deventerprisesoftware/scrapi-mcp"], "env": { "SCRAPI_API_KEY": "${env:SCRAPI_API_KEY}" } } } }

{ "mcpServers": { "scrapi": { "url": "http://localhost:5000/mcp" } } }

TRANSPORT=http PORT=5000 SCRAPI_API_KEY="your-scrapi-api-key" npx -y @deventerprisesoftware/scrapi-mcp

Windsurf Cascade stores MCP servers in~/.codeium/windsurf/mcp_config.json. You can also add servers fromWindsurf Settings>Cascade>MCP Servers. See theWindsurf MCP documentation.

{ "mcpServers": { "scrapi": { "command": "npx", "args": ["-y", "@deventerprisesoftware/scrapi-mcp"], "env": { "SCRAPI_API_KEY": "${env:SCRAPI_API_KEY}" } } } }

{ "mcpServers": { "scrapi": { "serverUrl": "http://localhost:5000/mcp" } } }

Windsurf supportsserverUrlorurlfor remote HTTP MCP servers. If your team uses enterprise MCP controls, the server ID in the admin whitelist must match the key name, for examplescrapi.

Kilo Code stores MCP configuration in the main Kilo config file. Use~/.config/kilo/kilo.jsoncfor global configuration,kilo.jsoncin the project root, or.kilo/kilo.jsoncfor project-specific configuration. See theKilo Code MCP documentation.

{ "mcp": { "scrapi": { "type": "local", "command": ["npx", "-y", "@deventerprisesoftware/scrapi-mcp"], "environment": { "SCRAPI_API_KEY": "your-scrapi-api-key" }, "enabled": true, "timeout": 300000 } } }

{ "mcp": { "scrapi": { "type": "remote", "url": "http://localhost:5000/mcp", "enabled": true, "timeout": 300000 } } }

On Windows, ifnpxis not found from the Kilo Code UI, usecmdas the command and pass/c,npx,-y, and@deventerprisesoftware/scrapi-mcpas arguments.

Codex supports MCP servers in the CLI and IDE extension. Both use the same MCP configuration. By default, Codex stores it in~/.codex/config.toml; trusted projects can also use.codex/config.toml. See theCodex MCP documentation.

codex mcp add scrapi --env SCRAPI_API_KEY="your-scrapi-api-key" -- npx -y @deventerprisesoftware/scrapi-mcp codex mcp list

Equivalentconfig.tomlstdio configuration:

[mcp_servers.scrapi] command = "npx" args = ["-y", "@deventerprisesoftware/scrapi-mcp"] startup_timeout_sec = 20 tool_timeout_sec = 300 [mcp_servers.scrapi.env] SCRAPI_API_KEY = "your-scrapi-api-key"

[mcp_servers.scrapi] url = "http://localhost:5000/mcp" tool_timeout_sec = 300

In the Codex terminal UI, run/mcpto confirm the server is connected.

VS Code stores MCP configuration in.vscode/mcp.jsonfor a workspace or in your user profile. The top-level key isservers, notmcpServers. See theVS Code MCP configuration reference.

{ "inputs": [ { "type": "promptString", "id": "scrapi-api-key", "description": "ScrAPI API key", "password": true } ], "servers": { "scrapi": { "type": "stdio", "command": "npx", "args": ["-y", "@deventerprisesoftware/scrapi-mcp"], "env": { "SCRAPI_API_KEY": "${input:scrapi-api-key}" } } } }

{ "servers": { "scrapi": { "type": "http", "url": "http://localhost:5000/mcp" } } }

Use the Command Palette commandsMCP: Add Server,MCP: List Servers, andMCP: Reset Cached Toolsto add, inspect, and refresh MCP servers.

Claude Code supports MCP servers through theclaude mcpCLI and the/mcpcommand inside Claude Code. See theClaude Code MCP documentation.

claude mcp add --transport stdio --env SCRAPI_API_KEY="your-scrapi-api-key" scrapi -- npx -y @deventerprisesoftware/scrapi-mcp claude mcp list

claude mcp add --transport http scrapi http://localhost:5000/mcp claude mcp list

To make the server available across all Claude Code projects, add--scope userbefore the server name:

claude mcp add --transport stdio --scope user --env SCRAPI_API_KEY="your-scrapi-api-key" scrapi -- npx -y @deventerprisesoftware/scrapi-mcp

Inside Claude Code, run/mcpto confirm the server is connected.

Use this shape for clients that accept a command, arguments, and environment variables:

{ "name": "ScrAPI", "command": "npx", "args": ["-y", "@deventerprisesoftware/scrapi-mcp"], "env": { "SCRAPI_API_KEY": "your-scrapi-api-key" } }

SetTRANSPORT=httpto run the server over Streamable HTTP.

TRANSPORT=http PORT=5000 SCRAPI_API_KEY="your-scrapi-api-key" npx -y @deventerprisesoftware/scrapi-mcp

$env:TRANSPORT = "http" $env:PORT = "5000" $env:SCRAPI_API_KEY = "your-scrapi-api-key" npx -y @deventerprisesoftware/scrapi-mcp

npx @modelcontextprotocol/inspector npx -y @deventerprisesoftware/scrapi-mcp

TRANSPORT=http PORT=5000 npx -y @deventerprisesoftware/scrapi-mcp

ScrAPI also provides hosted MCP endpoints:

Streamable HTTP: https://api.scrapi.tech/mcp

Cloud MCP servers are not yet supported by every MCP client. They are most useful for custom clients, MCP Inspector, or platforms that support remote MCP servers.

To authenticate with your ScrAPI API key, pass it as a query parameter or request header:

- Query parameter:https://api.scrapi.tech/mcp?apiKey=<YOUR_API_KEY>
- Request header:X-API-KEY: <YOUR_API_KEY>

The exact interaction depends on your MCP client. In most clients, you can either ask the model to use the ScrAPI tool or call the tool directly from a tool inspector.

{ "url": "https://example.com" }

Use ScrAPI to scrape https://example.com as Markdown and summarize the page.

{ "url": "https://example.com" }

Use ScrAPI to scrape https://example.com as HTML and extract every link.

ThebrowserCommandsvalue must be a string containing a JSON array.

{ "url": "https://example.com/search", "browserCommands": "[{\"input\":{\"input[name='q']\":\"web scraping\"}},{\"click\":\"button[type='submit']\"},{\"waitfor\":\"#results\"}]" }

{ "url": "https://example.com/products", "browserCommands": "[{\"scroll\":1200},{\"wait\":1000},{\"click\":\"button.load-more\"},{\"waitfor\":\".product-card:nth-child(25)\"}]" }
No reviews yet — be the first

Sign in to leave a review

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.

Install

Add to Claude Desktop or Cursor

{
    "mcpServers": {
        "scrapi mcp server": {
            "env": {
                "SCRAPI_API_KEY": "your-scrapi-api-key"
            },
            "args": [
                "-y",
                "@deventerprisesoftware/scrapi-mcp"
            ],
            "command": "npx"
        }
    }
}

More like this

Similar MCP servers you might find useful

Browse category

Sequential Thinking

Sequential Thinking is an MCP server that provides a tool for dynamic and reflective problem-solving through a structured, step-by-step thinking process. It is…

88.2k
Other

Filesystem

Filesystem is a Node.js server that implements the Model Context Protocol (MCP) for filesystem operations. It enables AI assistants to read, write, and manage…

88.1k
Other