MCP LLMS.txt Explorer
About
Explore and analyze websites that have implemented the llms.txt standard.
Details
- Author
- thedaviddias
- GitHub stars
- 75
- Downloads
- 354
- Categories
- Web Scraping, Other
Jump to
- Checks websites for llms.txt and llms-full.txt files
- Parses and validates llms.txt file contents
- Accesses structured data about compliant websites
- Provides check_website and list_websites tools
- Supports filtering by file type (llms.txt / llms-full.txt)
Setting up with Highlight
This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:
- Download and install Highlight from highlightai.com/download
- Navigate to the plugins tab and select "Add Custom Plugin"
-
Configure the plugin with the settings below
Plugin Name
MCP LLMS.txt ExplorerCommand (node, npx, python, etc.)Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
- Enable "Start Automatically" if you want the plugin to start when Highlight launches
From the repository
Install via Smithery (npx -y @smithery/cli install @thedaviddias/mcp-llms-txt-explorer --client claude) or manually (clone, pnpm install, pnpm run build). Configure in Claude Desktop’s JSON config. Use the tools check_website (domain URL → file locations and validation status) and list_websites (returns structured data, filterable by file type).
Claude Desktop / Cursor
Paste into your MCP client config file to install this server.
{
"mcpServers": {
"mcp llms.txt explorer": {
"mcp-llms-txt-explorer": {
"command": "npx",
"args": [
"-y",
"@smithery/cli",
"install",
"@thedaviddias/mcp-llms-txt-explorer",
"--client",
"claude"
]
}
}
}
}McpServers
{
"mcp-llms-txt-explorer": {
"command": "npx",
"args": [
"-y",
"@smithery/cli",
"install",
"@thedaviddias/mcp-llms-txt-explorer",
"--client",
"claude"
]
}
}MCP LLMS.txt Explorer
<a href="https://glama.ai/mcp/servers/lhyj3pva0z">
</a>
A Model Context Protocol server for exploring websites with llms.txt files. This server helps you discover and analyze websites that implement the llms.txt standard.
Features
Resources
- Check websites for llms.txt and llms-full.txt files - Parse and validate llms.txt file contents - Access structured data about compliant websitesTools
-check_website - Check if a website has llms.txt files
- Takes domain URL as input
- Returns file locations and validation status
- list_websites - List known websites with llms.txt files
- Returns structured data about compliant websites
- Supports filtering by file type (llms.txt/llms-full.txt)
Development
Install dependencies:
pnpm install
Build the server:
pnpm run build
For development with auto-rebuild:
pnpm run watch
Installation
Installing via Smithery
To install mcp-llms-txt-explorer for Claude Desktop automatically via Smithery:
npx -y @smithery/cli install @thedaviddias/mcp-llms-txt-explorer --client claude
Installing Manually
To use this server:```bash
Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
