WebSearch - Advanced Web Search and Content Extraction Tool

by m4s1t4

1 422 downloads Not rated yet

GitHub

About

An advanced web search and content extraction tool powered by the Firecrawl API for web scraping and analysis.

Explore

- Advanced Web Search: Perform intelligent web searches with customizable parameters
- Content Extraction: Extract specific information from web pages using natural language prompts
- Web Crawling: Crawl websites with configurable depth and limits
- Web Scraping: Scrape web pages with support for various output formats
- MCP Integration: Built as a Model Context Protocol (MCP) server for seamless integration

Setting up with Highlight

This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:

Download and install Highlight from highlightai.com/download
Navigate to the plugins tab and select "Add Custom Plugin"
Configure the plugin with the settings below

Plugin Name WebSearch - Advanced Web Search and Content Extraction Tool

Command (node, npx, python, etc.)

Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
Enable "Start Automatically" if you want the plugin to start when Highlight launches

Setting up with Cursor

Open Cursor's settings
Navigate to the MCP section
Click "Add new Global MCP server"
Copy and paste the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

Setting up with Claude Desktop

Open Claude Desktop's settings
Go to the Integrations tab
Click "Add MCP Server"
Use the configuration below based on your operating system

Note: Make sure Claude Desktop is running before attempting to connect.

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

Setting up with Windsurf

Open Windsurf's preferences
Go to the Integrations section
Click "Add MCP Integration"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

Setting up with Cline

Open Cline's settings
Navigate to the Integrations section
Click "Add MCP Server"
Copy and paste the configuration below based on your operating system

Note: Make sure to restart Cline after adding the configuration.

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

Setting up with Goose

Open Goose's preferences
Go to the Integrations tab
Select "MCP Servers"
Click "Add New Server"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

From the repository

- Python 3.8 or higher
- uv package manager
- Firecrawl API key
- OpenAI API key (optional, for enhanced features)
- Tavily API key (optional, for additional search capabilities)

1. Install uv:

uv venv

uv sync

5. Set up environment variables:

The tool requires certain API keys to function. We provide a .env.example file that you can use as a template: 1. Copy the example file:

bash

1. OpenAI API Key:

- Visit OpenAI's platform
- Sign up or log in
- Navigate to API keys section
- Create a new secret key

2. Firecrawl API Key:
- Visit Firecrawl's website
- Create an account
- Navigate to your dashboard
- Generate a new API key

If everything is configured correctly, you should receive a JSON response with search results.

1. Search

2. Extract Information

3. Crawl Websites

4. Scrape Content

Claude Desktop / Cursor

Paste into your MCP client config file to install this server.

{
    "mcpServers": {
        "websearch - advanced web search and content extraction tool": {
            "webSearch-Tools": {
                "command": "uv",
                "args": [
                    "venv"
                ]
            }
        }
    }
}

McpServers

{
    "webSearch-Tools": {
        "command": "uv",
        "args": [
            "venv"
        ]
    }
}

A powerful web search and content extraction tool built with Python, leveraging the Firecrawl API for advanced web scraping, searching, and content analysis capabilities.

🚀 Features

📋 Prerequisites

- Python 3.8 or higher
- uv package manager
- Firecrawl API key
- OpenAI API key (optional, for enhanced features)
- Tavily API key (optional, for additional search capabilities)

🛠️ Installation

1. Install uv:

```bash

No reviews yet — be the first

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.

About

Explore

Setting up with Highlight

From the repository

Claude Desktop / Cursor

McpServers

🚀 Features

📋 Prerequisites

🛠️ Installation

More like this

@mendableai/firecrawl Mcp Server

Sequential Thinking

Brave Search

Filesystem