MCP Smart Crawler
About
MCP Smart Crawler is a Model Context Protocol (MCP) server uses Playwright to crawl web content, extract metadata, and download resources such as videos and images.
Details
- License
- MIT
Explore
Extract metadata (title, description, images) from Xiaohongshu (小红书) posts.
Download videos and images from Xiaohongshu share links.
- Uses Playwright for browser automation.
Setting up with Highlight
This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:
- Download and install Highlight from highlightai.com/download
- Navigate to the plugins tab and select "Add Custom Plugin"
-
Configure the plugin with the settings below
Plugin Name
MCP Smart CrawlerCommand (node, npx, python, etc.)Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
- Enable "Start Automatically" if you want the plugin to start when Highlight launches
From the repository
To use this server with your MCP client, add the following configuration to your client's MCP server settings. You will need to adjust the command and args based on how you run the server script.
Example Configuration:
{
"mcpServers": {
"mcp-smart-crawler": {
"command": "npx",
"args": [
"-y",
"mcp-smart-crawler",
"--download-folder", // optional
"c:\\downloads" // optional
],
}
}
}
Example chat:
帮我查看这条小红书的内容和图片，并告诉我图片里面是什么
59 XXX发布了一篇小红书笔记，快来看吧！ 😆 OfTOBst2PsxctaX 😆 http://xhslink.com/a/xxaabbcczz，复制本条信息，打开【小红书】App查看精彩内容！
<br />
<hr />
<br />
<span id="mcp-smart-crawler-chinese"></span>
Claude Desktop / Cursor
Paste into your MCP client config file to install this server.
{
"mcpServers": {
"mcp smart crawler": {
"mcp-smart-crawler": {
"command": "npx",
"args": [
"-y",
"mcp-smart-crawler@latest",
"--download-folder",
"C:\\downloads"
]
}
}
}
}McpServers
{
"mcp-smart-crawler": {
"command": "npx",
"args": [
"-y",
"mcp-smart-crawler@latest",
"--download-folder",
"C:\\downloads"
]
}
}Key Features
Extract metadata (title, description, images) from Xiaohongshu (小红书) posts. Download videos and images from Xiaohongshu share links.- Uses Playwright for browser automation.
How to Use (MCP Client Configuration)
To use this server with your MCP client, add the following configuration to your client's MCP server settings. You will need to adjust the command and args based on how you run the server script.
Example Configuration:
{
"mcpServers": {
"mcp-smart-crawler": {
"command": "npx",
"args": [
"-y",
"mcp-smart-crawler",
"--download-folder", // optional
"c:\\downloads" // optional
],
}
}
}
Example chat:
帮我查看这条小红书的内容和图片，并告诉我图片里面是什么
59 XXX发布了一篇小红书笔记，快来看吧！ 😆 OfTOBst2PsxctaX 😆 http://xhslink.com/a/xxaabbcczz，复制本条信息，打开【小红书】App查看精彩内容！
<br />
<hr />
<br />
<span id="mcp-smart-crawler-chinese"></span>
Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
