Puppeteer Vision MCP Server - Specify4IT Configuration

This is a configured version of the Puppeteer Vision MCP server specifically set up for scraping specify4it.com. It includes custom settings and scripts for handling the site's specific structure, animations, and interactive elements.

Specific Features

- Configured for specify4it.com's structure and content
- Custom scripts for extracting:
- Page content and structure
- CSS animations and transitions
- JavaScript interactions
- Design elements and assets
- Organized output structure for website recreation

Now easily runnable via npx!

Features

- Scrapes webpages using Puppeteer with stealth mode
- Uses AI-powered interaction to automatically handle:
- Cookie consent banners
- CAPTCHAs
- Newsletter or subscription prompts
- Paywalls and login walls
- Age verification prompts
- Interstitial ads
- Any other interactive elements blocking content
- Extracts main content with Mozilla's Readability
- Converts HTML to well-formatted Markdown
- Special handling for code blocks, tables, and other structured content
- Accessible via the Model Context Protocol
- Option to view browser interaction in real-time by disabling headless mode
- Easily consumable as an npx package.

Quick Start with NPX

The recommended way to use this server is via npx, which ensures you're running the latest version without needing to clone or manually install.

1. Prerequisites: Ensure you have Node.js and npm installed.
2. Environment Setup:
The server requires an OPENAI_API_KEY. You can provide this and other optional configurations in two ways:
.env file: Create a .env file in the directory where you will run the npx command.
Shell Environment Variables: Export the variables in your terminal session.

Example .env file or shell exports:

    # Required

    OPENAI_API_KEY=your_api_key_here

# Optional (defaults shown)

    # VISION_MODEL=gpt-4.1

    # API_BASE_URL=https://api.openai.com/v1   # Uncomment to override

    # USE_SSE=true                             # Uncomment to use SSE mode instead of stdio

    # PORT=3001                                # Only used in SSE mode

    # DISABLE_HEADLESS=true                    # Uncomment to see the browser in action

    

3. Run the Server:
Open your terminal and run:

    npx -y puppeteer-vision-mcp-server

    

{
  "mcpServers": {
    "web-scraper": {
      "command": "npx",
      "args": ["-y", "puppeteer-vision-mcp-server"],
      "env": {
        "OPENAI_API_KEY": "YOUR_OPENAI_API_KEY_HERE",
        // Optional:
        // "VISION_MODEL": "gpt-4.1",
        // "API_BASE_URL": "https://api.example.com/v1",
        // "DISABLE_HEADLESS": "true" // To see the browser during operations
      }
    }
    // ... other MCP servers
  }
}

Example Success Response:

{

  "content": [

    {

      "type": "text",

      "text": "# Page Title\n\nThis is the content..."

    }

  ],

  "metadata": {

    "message": "Scraping successful",

    "success": true,

    "contentSize": 8734

  }

}

Example Error Response:*

{

  "content": [

    {

      "type": "text",

      "text": ""

    }

  ],

  "metadata": {

    "message": "Error scraping webpage: Failed to load the URL",

    "success": false

  }

}

How It Works

AI-Driven Interaction

Content Extraction

Installation & Development (for Modifying the Code)

If you wish to contribute, modify the server, or run a local development version:

1. Clone the Repository:

    git clone https://github.com/djannot/puppeteer-vision-mcp.git

    cd puppeteer-vision-mcp

    

    npm install

    

    npm run build

    

5. Run for Development:

    npm start # Starts the server using the local build

    

    npm run dev

    

Customization (for Developers)

You can modify the behavior of the scraper by editing:
- src/ai/vision-analyzer.ts (analyzePageWithAI function): Customize the AI prompt.
- src/ai/page-interactions.ts (executeAction function): Add new action types.
- src/scrapers/webpage-scraper.ts (visitWebPage function): Change Puppeteer options.
- src/utils/markdown-formatters.ts: Adjust Turndown rules for Markdown conversion.

Dependencies