Browserless

by browserless

2 stars
431 downloads
Not rated
GitHub

About

Browserless is an MCP (Model Context Protocol) server that exposes the Browserless.io smart scraper API to LLM clients like Claude Desktop, Cursor, VS Code, and Windsurf. It allows AI agents to scrape, crawl, search, audit, and automate web pages directly.

Details

Author
browserless
GitHub stars
2
Downloads
431
Categories
Web Scraping, Automation, Other

- Scrape JavaScript-heavy pages with anti-bot handling automatically.
- Crawl entire websites with depth control and path filtering.
- Search the web and scrape results with geo-targeting filters.
- Run Lighthouse audits for accessibility, performance, and SEO.
- Execute custom Puppeteer JavaScript in the cloud.
- Drive persistent browser sessions via ReAct loop with skill library.

Setting up with Highlight

This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:

  1. Download and install Highlight from highlightai.com/download
  2. Navigate to the plugins tab and select "Add Custom Plugin"
  3. Configure the plugin with the settings below
    Plugin Name Browserless
    Command (node, npx, python, etc.)

    Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.

  4. Enable "Start Automatically" if you want the plugin to start when Highlight launches

From the repository

Obtain an API token from browserless.io (free tier available). Configure your MCP client with the URL https://mcp.browserless.io/mcp?token=your-token-here or use the Authorization: Bearer your-token-here header. No local installation is required; the server is hosted.

browserless_export

Export a webpage from a URL via the Browserless /export API. Fetches the URL and returns its content in the native format (HTML, PDF, image, etc.). Automatically detects the content type. Set includeResources=true to bundle all page assets (CSS, JS, images) into a ZIP archive for offline use.

browserless_skill

Load a Browserless agent skill on demand, or discover site-specific recipes. Two uses: - **{ site: "<host>" }** — list any **site-specific recipes** tuned for that host (e.g. `{ site: "ebay.com" }`), returned as pointers. Do this as soon as you know the host you're about to drive; if one matches your task, load it by id. Returns a "no recipe" note when there's none. - **{ id: "<id>" }** — load a skill body: an in-house skill id (list below) OR a site recipe id `host/slug` from a `site` lookup. Use the in-house skills below when you suspect the page exhibits a non-trivial mechanic but no SKILL block was auto-injected. The auto-injection heuristics are conservative; calling this tool is the explicit fallback. Available in-house skills: - **shadow-dom** — deep selectors, iframe URL-pattern syntax, what works through deep-ref - **cookie-consent** — vendor-specific dismiss recipes (OneTrust, Cookiebot, Didomi, etc.) - **modals** — close-button heuristics, ESC handling, alertdialog vs. dialog - **snapshot-misses** — truncated/empty snapshots, image-rendered content - **dynamic-content** — choosing the right `wait*` method after async triggers - **screenshots** — when to screenshot vs. snapshot, scope and format choices - **tabs** — multi-tab workflows, peek-without-switching - **autonomous-login** — load before authenticating: when the user asked you to log in, when a wall blocks the task, or as soon as a password input appears. Covers the don't-login-by-default posture, contextual credential matching, MFA/captcha branches, and the required final JSON response shape. - **captchas** — the `solve` command, response semantics, escalation path (Cloud-only) - **file-transfers** — `uploadFile` / `getDownloads`, stdio-path vs. base64 content, size caps

browserless_agent

READ CAREFULLY: Execute browser commands in persistent agent session. ## Core Loop (ReAct: Reason → Act → Observe) 0. **Plan + check for a site recipe** — restate the goal, decide the target host, then `browserless_skill { site: "<host>" }` (see above). Load and follow any matching recipe before writing your own plan. Never jump straight to `goto`. 1. **goto** — waits "domcontentloaded" 2. **snapshot** — returns interactive + informational elements (button, link, textbox, combobox, checkbox, heading, img+alt) with ref= selectors 3. **Plan** all actions from snapshot 4. **Batch** execute 5. **Re-snapshot** only if page changed 6. Repeat → **close** when done ## Ending the session (REQUIRED) An open session holds one of the account's concurrent browsers until it idles out — leaving it open is not free, and stacking them starves the next task. - **Task complete? Close it.** Send `{ "method": "close" }` as its own call, as the last thing you do. This is the default for one-shot work (a lookup, a scrape, a form submit): close without asking. - **Ask instead of guessing** only when follow-up in the SAME browser is genuinely likely (the user said "then...", you're mid-flow on a logged-in site, or the result invites a next step). Say the browser is still open, ask whether to close it, and close it as soon as they're done. - **Never** end your reply with a live session and no mention of it. Either it's closed, or you told the user it's open and why. ## Site recipes (site-specific, NOT auto-injected) — CHECK FIRST Many specific sites (marketplaces, gov portals, travel, real-estate, etc.) have a **tuned recipe** for a given task — proven selectors, API shortcuts, proxy needs, and known gotchas that a from-scratch plan will miss. These are **not** auto-injected; you must ask for them, and a recipe **overrides** any plan you'd build yourself (including "just use a prefiltered URL + evaluate"). **This is step 0 of every task — do it before your first `goto`.** The moment you know the target host (the user named the site, or you resolved which site to use), call `browserless_skill { site: "<host>" }` — e.g. `{ site: "airbnb.com" }`. If it lists a recipe matching your task, load it with `browserless_skill { id: "<host>/<slug>" }` and follow it. Only when there's no match do you plan the steps yourself. Skipping this check on a supported site is a mistake — it's one cheap call. **Report the outcome (only if you loaded a site recipe).** As your final command in the run, send `{ method: "reportSkillOutcome", params: { domain: "<host>", task: "<slug>", success: <bool> } }` inside `commands` — where `domain`/`task` are the loaded recipe's `<host>`/`<slug>` and `success` is whether the recipe actually got you the result. This refines shared recipes and retires ones that stop working. Send it once, and only when you loaded a recipe — never for a self-planned run. Send it as your last command **before** any `close` (close ends the run and anything after it is dropped). ## Proxy (optional) Proxy config is a **top-level tool argument** (`proxy`, `proxyCountry`, etc. on the tool call itself) — it is applied when the session is opened. **NEVER call `proxy` as a method inside `commands`** — a `{ method: "proxy", ... }` JSON-RPC mutation does NOT change the upstream proxy on an already-open session and will silently no-op. **If there is credible evidence the task needs a proxy, you MUST pass proxy options on the very FIRST call** (before any `goto`/`snapshot`), because the config is read once at session creation. Credible signals include: the user asks for a specific country/region/locale; the target site is known to geo-restrict or block datacenter IPs (streaming, ticketing, retail, banking, real-estate, news paywalls); a prior attempt returned 403/451/captcha/"unusual traffic"/"access denied"; the user explicitly mentions residential / sticky IP / proxy. If you already opened a session without a proxy and now realize one is needed, you must `close`…

browserless_search

Search the web using Browserless and optionally scrape each result. Performs web searches via SearXNG and can return results from web, news, or images. Optionally scrape each result URL to get markdown, HTML, links, or screenshots. Useful for research, gathering information, and finding relevant web pages.

browserless_performance

Run a Lighthouse performance audit on any URL via the Browserless /performance API. Returns scores and metrics for accessibility, best practices, performance, PWA, and SEO. Optionally filter by category or supply performance budgets. Note: audits can take 30s–120s depending on the site.

browserless_account

Read the Browserless account behind the current API token: plan, unit balance, billing period, and the names of the account API keys. Use it to answer "what plan am I on", "how many units are left", or "which keys exist". Read-only, and never returns API token values.

browserless_usage

Read request and unit consumption for the Browserless account behind the current API token: successes, errors, timeouts, queueing, peak concurrency, captchas, proxy bytes and units. Use it to answer "how much have I used" or "why is my bill high". For per-request detail on failures, use browserless_logs instead. Read-only.

browserless_sessions

Inspect the sessions on the Browserless account behind the current API token: browsers running right now, persistent sessions saved on dedicated workers, recorded session replays, and 1Password credential integrations. Use it to answer "what is running", "did my session survive", or "what got recorded". Read-only — it never stops a session. Action `replay` downloads one recording and returns a fully self-contained playable rrweb page, needing no network to render: display it inline if you can render HTML, otherwise build an artifact from the returned instructions so the user can watch it. Always show the replay — never just summarise it in words.

browserless_logs

Read Browserless's own record of the account's recent requests: what was attempted, whether it failed, why it stopped, how long it took and what it cost. This is the tool for diagnosing a run that failed on the Browserless side rather than in your own code. The window available depends on the account plan; the server reports the limit if a range is refused. Read-only.

browserless_smartscraper

Scrape a SINGLE webpage and return its content as markdown or HTML. Handles JavaScript-heavy pages and anti-bot measures automatically. For content across MULTIPLE pages of a site, use browserless_crawl; to list a site's URLs, use browserless_map.

browserless_function

Execute custom Puppeteer JavaScript code on the Browserless cloud. Your function receives a Puppeteer `page` object and optional `context` data. Return { data, type } to control the response payload and Content-Type. For binary outputs, set `type` to a real MIME so the bytes come back as a proper content block instead of base64 text: - `image/png` / `image/jpeg` / `image/webp` → vision content block (~1.5K tokens) - `audio/mpeg` / `audio/wav` → audio content block - `application/pdf` and other binaries → resource content block (attachment) Text responses are capped at 200,000 characters (~50K tokens). Larger text payloads will be rejected — filter or summarize inside your function, or switch to a binary type if you actually meant to return bytes. Useful for complex scraping, form filling, or any browser automation that requires custom code.

browserless_map

Discover and map all URLs on a website using Browserless. Scans a site via sitemaps and link extraction to find all pages. Returns a list of URLs with optional titles and descriptions. Use the search parameter to order results by relevance to a query. Useful for site audits, content discovery, and building site maps.

browserless_crawl

Crawl a website and scrape every discovered page using Browserless. Starts from a seed URL and follows links up to a configurable depth. Supports sitemap discovery, path filtering, subdomain handling, and custom scrape options. Returns scraped content (markdown/HTML) for each page along with metadata. Useful for comprehensive site analysis, content extraction, and data gathering.

browserless_profiles

List the authentication profiles saved for the current token. A profile is a saved logged-in browser state (cookies + storage) that can be replayed by passing its name as `profile` to other tools. Call this before a task that needs the browser to start signed in, to discover which profiles already exist and pick one by name. Returns each profile name plus cookie/origin counts and last-used time.

Claude Desktop / Cursor

Paste into your MCP client config file to install this server.

{
    "mcpServers": {
        "browserless": {
            "browserless": {
                "type": "http",
                "url": "https://mcp.browserless.io/mcp",
                "headers": {
                    "Authorization": "Bearer your-token-here",
                    "x-browserless-api-url": "https://production-sfo.browserless.io"
                }
            }
        }
    }
}

McpServers

{
    "browserless": {
        "type": "http",
        "url": "https://mcp.browserless.io/mcp",
        "headers": {
            "Authorization": "Bearer your-token-here",
            "x-browserless-api-url": "https://production-sfo.browserless.io"
        }
    }
}

Browserless MCP Server

MCP Badge

MCP (Model Context Protocol) server for Browserless.io — expose the Browserless smart scraper API to LLM clients like Claude Desktop, Cursor, VS Code, and Windsurf.

Quick Start

Get an API token from browserless.io (free tier available), then point your MCP client at the hosted server:

{
  "mcpServers": {
    "browserless": {
      "url": "https://mcp.browserless.io/mcp?token=your-token-here"
    }
  }
}

No local install — see Configuration for per-client snippets.

Tools

| Tool | Description |
| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| browserless_smartscraper | Scrape a single webpage and return its content as markdown or HTML. Handles JavaScript-heavy pages and anti-bot measures automatically. For content across multiple pages, use browserless_crawl; to list a site's URLs, use browserless_map. |
| browserless_search | Search the web using Browserless and optionally scrape each result. Supports web, news, and image search with geo-targeting and time filters. |
| browserless_map | Discover and map all URLs on a website. Scans via sitemaps and link extraction. Returns URLs with optional titles and descriptions. Useful for site audits and content discovery. |
| browserless_crawl | Crawl a website and scrape every discovered page. Supports depth control, path filtering, sitemap strategies, and configurable scrape options. Returns scraped content and metadata for each page. |
| browserless_performance | Run Lighthouse audits on any URL. Returns scores and metrics for accessibility, best practices, performance, PWA, and SEO. Optionally filter by category or supply performance budgets. |
| browserless_function | Execute custom Puppeteer JavaScript on the Browserless cloud. The function receives a page object and optional context; return { data, type } to control the payload and Content-Type. |
| browserless_export | Export a webpage via the Browserless /export API. Fetches the URL and returns its native content (HTML, PDF, image, etc.) with automatic content-type detection. |
| browserless_agent | Drive a persistent browser session via a ReAct loop: snapshot the page, plan, batch interactions (click, type, scroll, evaluate, etc.), and re-snapshot. Uses ref-based selectors derived from snapshots, supports multi-tab workflows, screenshots, captcha solving, live URLs, and file upload/download (captured downloads auto-surface as handles; bytes never enter context). |
| browserless_skill | Load an on-demand recipe for a non-trivial page mechanic (shadow DOM, cookie consent, modals, captchas, dynamic content, snapshot misses, screenshots, tabs). Companion to browserless_agent. |

Skills

The server ships with a built-in library of Skills — on-demand recipes the agent can load to handle tricky page mechanics. Skills auto-inject into browserless_agent responses when their triggers fire (e.g. the agent hits a cookie banner), and can also be loaded manually via the browserless_skill tool.

| Skill | Source | Purpose |
| ----------------- | -------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| shadow-dom | src/skills/shadow-dom.md | Deep selectors and iframe targeting through shadow roots. |
| cookie-consent | src/skills/cookie-consent.md | Vendor-specific dismiss recipes (OneTrust, Cookiebot, Didomi, TrustArc, etc.). |
| modals | src/skills/modals.md | Closing dialogs, alertdialogs, and overlay close-button heuristics. |
| captchas | src/skills/captchas.md | Using the solve command, response semantics, and escalation paths (Cloud only). |
| dynamic-content | src/skills/dynamic-content.md | Choosing the right wait method for async/AJAX/SPA content. |
| snapshot-misses | src/skills/snapshot-misses.md | Handling truncated/empty snapshots and image-rendered content. |
| screenshots | src/skills/screenshots.md | When to screenshot vs. snapshot, scope and format choices. |
| tabs | src/skills/tabs.md | Multi-tab workflows and peek-without-switching via targetId. |

Load a skill explicitly:

{
  "method": "tools/call",
  "params": {
    "name": "browserless_skill",
    "arguments": { "id": "cookie-consent" },
  },
}

Residential proxy (browserless_agent)

Pass a top-level proxy object on browserless_agent to route the session through residential IPs. Use this when targets IP-block datacenter traffic.

{
  "method": "tools/call",
  "params": {
    "name": "browserless_agent",
    "arguments": {
      "method": "goto",
      "params": { "url": "https://example.com" },
      "proxy": {
        "proxy": "residential",
        "proxyCountry": "us",
        "proxySticky": true,
      },
    },
  },
}

| Field | Notes |
| --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| proxy | "residential" — only value supported today. |
| proxyCountry | ISO-2 country code ("us", "de"). Auto-normalized to lowercase. Non-letter values are rejected. |
| proxyState | US state name with whitespace replaced by underscores ("new_york"). Paid-plan gated — non-eligible tokens get a 401. |
| proxyCity | City target. Paid/enterprise plan gated — non-eligible tokens get a 401. |
| proxySticky | Stable IP while the underlying WebSocket stays open. Reconnects (idle drop, network blip, browser crash) allocate a new sticky id and new IP. |
| proxyLocaleMatch | Match navigator locale to the proxy IP country. |
| proxyPreset | Named preset (e.g. "px_amazon01"). Available presets are plan-dependent — ask Browserless support for your list. |
| externalProxyServer | Bring-your-own upstream, e.g. http://user:pass@host:port. Must be http:// or https://. |

> Note: proxyCountry / proxyState / proxyCity / proxySticky / proxyLocaleMatch / proxyPreset require either proxy: "residential" or externalProxyServer to be set. The MCP rejects this combination at validation time; without it, the API would silently ignore them.

The proxy object is read once at session creation. To change it, call close and start a new session — the agent client keys sessions on the proxy fingerprint, so passing a different config will land on a fresh WebSocket.

Configuration

The server is hosted at https://mcp.browserless.io/mcp. Authenticate via headers (preferred) or a ?token= query parameter.

Installing via an AI agent? See install.md for agent-readable setup instructions.

Using headers (recommended for clients that support them):

{
  "mcpServers": {
    "browserless": {
      "url": "https://mcp.browserless.io/mcp",
      "headers": {
        "Authorization": "Bearer your-token-here"
      }
    }
  }
}

Using URL query parameters (for clients like Claude.ai custom connectors that only accept a URL):

https://mcp.browserless.io/mcp?token=your-token-here

To connect to a specific Browserless regional endpoint, add the x-browserless-api-url header or the browserlessUrl query parameter:

{
  "mcpServers": {
    "browserless": {
      "url": "https://mcp.browserless.io/mcp",
      "headers": {
        "Authorization": "Bearer your-token-here",
        "x-browserless-api-url": "https://production-lon.browserless.io"
      }
    }
  }
}
https://mcp.browserless.io/mcp?token=your-token-here&browserlessUrl=https://production-lon.browserless.io

When both headers and query parameters are present, headers take precedence.

Claude Desktop

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "browserless": {
      "url": "https://mcp.browserless.io/mcp?token=your-token-here"
    }
  }
}

Cursor

Add to your Cursor MCP settings:

{
  "mcpServers": {
    "browserless": {
      "url": "https://mcp.browserless.io/mcp?token=your-token-here"
    }
  }
}

VS Code

Add to your VS Code settings (settings.json):

{
  "mcp": {
    "servers": {
      "browserless": {
        "url": "https://mcp.browserless.io/mcp",
        "headers": {
          "Authorization": "Bearer your-token-here"
        }
      }
    }
  }
}

Windsurf

Add to your Windsurf MCP configuration:

{
  "mcpServers": {
    "browserless": {
      "url": "https://mcp.browserless.io/mcp?token=your-token-here"
    }
  }
}

Self-Hosting

The server can also be run locally — useful for air-gapped deployments or pointing at a self-hosted Browserless instance. Clone this repo and build the Docker image:

docker build -f docker/Dockerfile -t browserless-mcp .

docker run \
-e BROWSERLESS_TOKEN=your-token \
-e BROWSERLESS_API_URL=https://your-browserless-instance.example.com \
-p 8080:8080 \
browserless-mcp

Then point your MCP client at http://localhost:8080/mcp using the same header/query-parameter auth as above.

Self-hosted environment variables

| Variable | Required | Default | Description |
| ------------------------- | -------- | --------------------------------------- | -------------------------------------------------- |
| BROWSERLESS_TOKEN | Yes | — | Your Browserless API token |
| BROWSERLESS_API_URL | No | https://production-sfo.browserless.io | API endpoint (for self-hosted Browserless) |
| TRANSPORT | No | stdio | Transport type: stdio or httpStream |
| PORT | No | 8080 | HTTP server port (only for httpStream transport) |
| BROWSERLESS_TIMEOUT | No | 30000 | Request timeout in milliseconds |
| BROWSERLESS_MAX_RETRIES | No | 3 | Max retry attempts for failed requests |
| BROWSERLESS_CACHE_TTL | No | 60000 | Cache TTL in milliseconds (0 to disable) |

MCP Resources

| Resource URI | Description |
| ------------------------ | ------------------------------- |
| browserless://api-docs | Smart scraper API documentation |
| browserless://status | Live service health status |

MCP Prompts

| Prompt | Description |
| ----------------- | ------------------------------------------- |
| scrape-url | Scrape a webpage and summarize its content |
| extract-content | Extract specific information from a webpage |

Development

npm install
npm run build
npm test
npm run coverage

Tests

The test suite uses Mocha with Chai and Sinon. Specs live alongside the code in test/ (test/lib/, test/tools/, test/prompts/, test/resources/, test/integration/) and run against the compiled output in build/.

- npm test — compiles TypeScript and runs every .spec.js under build/test/. No external services or BROWSERLESS_TOKEN are required; the API client is stubbed.
- npm run coverage — runs the suite under c8 with the thresholds configured in package.json (lines ≥ 80%, branches ≥ 70%, functions ≥ 80%).

Tests run automatically on every pull request via the Test workflow on Node 24. PRs must keep the suite green before they can merge.

API Token

Get your API token at browserless.io. The token authenticates all requests to the Browserless API.

License

SSPL-1.0

No reviews yet — be the first

Sign in to leave a review

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.