UseScraper MCP Server

by tanevanwifferen

1 stars

1 downloads

Not rated

GitHub

About

A TypeScript-based MCP server utilizing the UseScraper API to provide web scraping capabilities, allowing users to extract content from webpages in various formats.

Details

Author: tanevanwifferen
Repository: tanevanwifferen/usescraper-mcp-server
GitHub stars: 1
Downloads: 1
License: MIT License
Categories: Web Scraping, Automation, API, Search, Developer Tools, Other

Setting up with Highlight

Follow these steps to add this server as a custom Highlight plugin:

Download and install Highlight from highlightai.com/download
Navigate to the plugins tab and select "Add Custom Plugin"
Configure the plugin with the settings below
Plugin Name UseScraper MCP Server

Command (node, npx, python, etc.) node
Arguments
- Argument 1 /path/to/usescraper-server/build/index.js
Environment
- USESCRAPER_API_KEY your-api-key-here
Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
Enable "Start Automatically" if you want the plugin to start when Highlight launches

Setting up with Cursor

Open Cursor's settings
Navigate to the MCP section
Click "Add new Global MCP server"
Copy and paste the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

Setting up with Claude Desktop

Open Claude Desktop's settings
Go to the Integrations tab
Click "Add MCP Server"
Use the configuration below based on your operating system

Note: Make sure Claude Desktop is running before attempting to connect.

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

Setting up with Windsurf

Open Windsurf's preferences
Go to the Integrations section
Click "Add MCP Integration"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

Setting up with Cline

Open Cline's settings
Navigate to the Integrations section
Click "Add MCP Server"
Copy and paste the configuration below based on your operating system

Note: Make sure to restart Cline after adding the configuration.

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

Setting up with Goose

Open Goose's preferences
Go to the Integrations tab
Select "MCP Servers"
Click "Add New Server"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

JSON

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

From the repository

Once configured, you can use the 'scrape' tool through the MCP interface. Example usage:

{
  "name": "scrape",
  "arguments": {
    "url": "https://example.com",
    "format": "markdown"
  }
}

To install UseScraper for Claude Desktop automatically via Smithery:

npx -y @smithery/cli install usescraper-server --client claude

1. Clone the repository:

   git clone https://github.com/your-repo/usescraper-server.git

   cd usescraper-server

2. Install dependencies:

   npm install

3. Build the server:

   npm run build

`scrape`

Extract content from a webpage. Parameters: url (required), format (optional: text, html, markdown; Default: markdown), advanced_proxy (optional: boolean; Default: false), extract_object (optional: object specifying data to extract)

- scrape - Extract content from a webpage
- Parameters:
- url (required): The URL of the webpage to scrape
- format (optional): The format to save the content (text, html, markdown). Default: markdown
- advanced_proxy (optional): Use advanced proxy to circumvent bot detection. Default: false
- extract_object (optional): Object specifying data to extract

Claude Desktop / Cursor

Paste into your MCP client config file to install this server.

{
    "mcpServers": {
        "usescraper mcp server": {
            "env": {
                "USESCRAPER_API_KEY": "your-api-key-here"
            },
            "args": [
                "/path/to/usescraper-server/build/index.js"
            ],
            "command": "node"
        }
    }
}

Linux

{
    "env": {
        "USESCRAPER_API_KEY": "your-api-key-here"
    },
    "args": [
        "/path/to/usescraper-server/build/index.js"
    ],
    "command": "node"
}

Macos

{
    "env": {
        "USESCRAPER_API_KEY": "your-api-key-here"
    },
    "args": [
        "/path/to/usescraper-server/build/index.js"
    ],
    "command": "node"
}

Windows

{
    "env": {
        "USESCRAPER_API_KEY": "your-api-key-here"
    },
    "args": [
        "/path/to/usescraper-server/build/index.js"
    ],
    "command": "node"
}

UseScraper MCP Server

This is a TypeScript-based MCP server that provides web scraping capabilities using the UseScraper API. It exposes a single tool 'scrape' that can extract content from web pages in various formats.

Features

Tools

- scrape - Extract content from a webpage - Parameters: - url (required): The URL of the webpage to scrape - format (optional): The format to save the content (text, html, markdown). Default: markdown - advanced_proxy (optional): Use advanced proxy to circumvent bot detection. Default: false - extract_object (optional): Object specifying data to extract

Installation

Installing via Smithery

To install UseScraper for Claude Desktop automatically via Smithery:

npx -y @smithery/cli install usescraper-server --client claude

Manual Installation

1. Clone the repository:

   git clone https://github.com/your-repo/usescraper-server.git
   cd usescraper-server

2. Install dependencies:

   npm install

3. Build the server:

   npm run build

Configuration

To use with Claude Desktop, add the server config:

On MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json
On Windows: %APPDATA%/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "usescraper-server": {
      "command": "node",
      "args": ["/path/to/usescraper-server/build/index.js"],
      "env": {
        "USESCRAPER_API_KEY": "your-api-key-here"
      }
    }
  }
}

Replace /path/to/usescraper-server with the actual path to the server and your-api-key-here with your UseScraper API key.

Usage

Once configured, you can use the 'scrape' tool through the MCP interface. Example usage:

{
  "name": "scrape",
  "arguments": {
    "url": "https://example.com",
    "format": "markdown"
  }
}

Development

For development with auto-rebuild:

npm run watch

Debugging

Since MCP servers communicate over stdio, debugging can be challenging. We recommend using the MCP Inspector, which is available as a package script:

npm run inspector

The Inspector will provide a URL to access debugging tools in your browser.

No reviews yet — be the first

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.

About

Details

Jump to

Setting up with Highlight

From the repository

scrape

Claude Desktop / Cursor

Linux

Macos

Windows

UseScraper MCP Server

Features

Tools

Installation

Installing via Smithery

Manual Installation

Configuration

Usage

Development

Debugging

Install

More like this

@mendableai/firecrawl Mcp Server

Umut Naci

Playwright

Fetch

GitLab

Puppeteer

`scrape`