WebSearch - Advanced Web Search and Content Extraction Tool

by m4s1t4

1 stars
422 downloads
Not rated
GitHub

About

An advanced web search and content extraction tool powered by the Firecrawl API for web scraping and analysis.

Details

Author
m4s1t4
GitHub stars
1
Downloads
422
Categories
Search, Web Scraping, Other

- Advanced Web Search with customizable parameters
- Content extraction using natural language prompts
- Web crawling with configurable depth and limits
- Web scraping supporting multiple output formats
- MCP integration for seamless tool access

Setting up with Highlight

This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:

  1. Download and install Highlight from highlightai.com/download
  2. Navigate to the plugins tab and select "Add Custom Plugin"
  3. Configure the plugin with the settings below
    Plugin Name WebSearch - Advanced Web Search and Content Extraction Tool
    Command (node, npx, python, etc.)

    Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.

  4. Enable "Start Automatically" if you want the plugin to start when Highlight launches

From the repository

Install the uv package manager, clone the repository, create a virtual environment, install dependencies with uv sync, and set required environment variables (Firecrawl API key, optionally OpenAI and Tavily keys). Then configure the MCP server in Claude for Desktop’s claude_desktop_config.json by adding the server command pointing to main.py. The server exposes four tools: Search, Extract, Crawl, and Scrape.

Claude Desktop / Cursor

Paste into your MCP client config file to install this server.

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

McpServers

{
    "webSearch-Tools": {
        "command": "uv",
        "args": [
            "venv"
        ]
    }
}

WebSearch - Advanced Web Search and Content Extraction Tool

License
Python Version
Firecrawl
uv

A powerful web search and content extraction tool built with Python, leveraging the Firecrawl API for advanced web scraping, searching, and content analysis capabilities.

🚀 Features

- Advanced Web Search: Perform intelligent web searches with customizable parameters
- Content Extraction: Extract specific information from web pages using natural language prompts
- Web Crawling: Crawl websites with configurable depth and limits
- Web Scraping: Scrape web pages with support for various output formats
- MCP Integration: Built as a Model Context Protocol (MCP) server for seamless integration

📋 Prerequisites

- Python 3.8 or higher
- uv package manager
- Firecrawl API key
- OpenAI API key (optional, for enhanced features)
- Tavily API key (optional, for additional search capabilities)

🛠️ Installation

1. Install uv:

```bash

No reviews yet — be the first

Sign in to leave a review

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.