UI Explorer MCP Server
About
UI Explorer MCP Server is an MCP server that provides tools for exploring and interacting with UI elements on your screen. It is designed for developers and testers who need to automate UI workflows on Windows.
Details
- License
- MIT license
Explore
- Explore UI hierarchies: Scan and analyze all UI elements on your screen
- Screenshot UI with highlights: Visualize UI elements with boundaries and hierarchy
- Control mouse clicks: Click on UI elements based on coordinates
- Explore specific regions: Focus on parts of the screen like top-left, center, etc.
- 🎬 Macro Recording: Record and playback UI workflows independently or through MCP
Setting up with Highlight
This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:
- Download and install Highlight from highlightai.com/download
- Navigate to the plugins tab and select "Add Custom Plugin"
-
Configure the plugin with the settings below
Plugin Name
UI Explorer MCP ServerCommand (node, npx, python, etc.)Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
- Enable "Start Automatically" if you want the plugin to start when Highlight launches
From the repository
- Windows operating system
- Python 3.10+
- MCP 1.6.0+
- PyAutoGUI
- PyWinAuto
- Pillow
- Pydantic 2.0+
New! You can now record UI workflows without running the MCP server:
```bash
After installing the package, you need to configure your MCP client to use the UI Explorer server.
The UI Explorer provides several MCP tools for UI automation and exploration. These tools are available when using the MCP server interface.
Claude Desktop / Cursor
Paste into your MCP client config file to install this server.
{
"mcpServers": {
"ui explorer mcp server": {
"mcp-ui-explorer": {
"command": "python",
"args": [
"record_macro.py",
"--name",
"Login Workflow"
]
}
}
}
}McpServers
{
"mcp-ui-explorer": {
"command": "python",
"args": [
"record_macro.py",
"--name",
"Login Workflow"
]
}
}An MCP server that provides tools for exploring and interacting with UI elements on your screen.
Features
- Explore UI hierarchies: Scan and analyze all UI elements on your screen
- Screenshot UI with highlights: Visualize UI elements with boundaries and hierarchy
- Control mouse clicks: Click on UI elements based on coordinates
- Explore specific regions: Focus on parts of the screen like top-left, center, etc.
- 🎬 Macro Recording: Record and playback UI workflows independently or through MCP
🚀 Quick Start: Standalone Macro Recording
New! You can now record UI workflows without running the MCP server:
```bash
Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
