Computer Control MCP

MCP server that provides computer control capabilities, like mouse, keyboard, OCR, etc. using PyAutoGUI, RapidOCR, ONNXRuntime. Similar to 'computer-use' by Anthropic. With Zero External Dependencies.

Note:Runninguvx computer-control-mcp@latestfor the first time will download python dependencies (around 70MB) which may take some time. Recommended to run this in a terminal before using it as MCP. Subsequent runs will be instant.

{ "mcpServers": { "computer-control-mcp": { "command": "uvx", "args": ["computer-control-mcp@latest"] } } }

computer-control-mcp # instead of uvx computer-control-mcp, so you can use the latest version, also you can uv cache clean to clear the cache and uvx again to use latest version.

- Control mouse movements and clicks
- Type text at the current cursor position
- Take screenshots of the entire screen or specific windows with optional saving to downloads directory
- Extract text from screenshots using OCR (Optical Character Recognition)
- List and activate windows
- Press keyboard keys
- Drag and drop operations
- Enhanced screenshot capture for GPU-accelerated windows (Windows only)

Traditional screenshot methods like GDI/PrintWindow fail to capture GPU-accelerated windows, resulting in black screens. This impacts games, media players, Electron apps, browsers with GPU acceleration, streaming software, and CAD tools. Use WGC throughtake_screenshottool's flag orENVvariable

By default, screenshots are saved to the OS downloads directory. You can customize this by setting theCOMPUTER_CONTROL_MCP_SCREENSHOT_DIRenvironment variable:

{ "mcpServers": { "computer-control-mcp": { "command": "uvx", "args": ["computer-control-mcp@latest"], "env": { "COMPUTER_CONTROL_MCP_SCREENSHOT_DIR": "C:\\Users\\YourName\\Pictures\\Screenshots" } } } }

# Windows (PowerShell) $env:COMPUTER_CONTROL_MCP_SCREENSHOT_DIR = "C:\Users\YourName\Pictures\Screenshots" # macOS/Linux export COMPUTER_CONTROL_MCP_SCREENSHOT_DIR="/home/yourname/Pictures/Screenshots"

If the specified directory doesn't exist, the server will fall back to the default downloads directory.

You can configure the system to automatically use Windows Graphics Capture (WGC) for specific windows by setting theCOMPUTER_CONTROL_MCP_WGC_PATTERNSenvironment variable. This variable should contain comma-separated patterns that match window titles:

{ "mcpServers": { "computer-control-mcp": { "command": "uvx", "args": ["computer-control-mcp@latest"], "env": { "COMPUTER_CONTROL_MCP_WGC_PATTERNS": "obs, discord, game, steam" } } } }

# Windows (PowerShell) $env:COMPUTER_CONTROL_MCP_WGC_PATTERNS = "obs, discord, game, steam" # macOS/Linux export COMPUTER_CONTROL_MCP_WGC_PATTERNS="obs, discord, game, steam"

When this variable is set, any window whose title contains any of the specified patterns will automatically use WGC for screenshot capture, eliminating black screens for GPU-accelerated applications.

- click_screen(x: int, y: int): Click at specified screen coordinates
- move_mouse(x: int, y: int): Move mouse cursor to specified coordinates
- drag_mouse(from_x: int, from_y: int, to_x: int, to_y: int, duration: float = 0.5): Drag mouse from one position to another
- mouse_down(button: str = "left"): Hold down a mouse button ('left', 'right', 'middle')
- mouse_up(button: str = "left"): Release a mouse button ('left', 'right', 'middle')

- type_text(text: str): Type the specified text at current cursor position
- key_down(key: str): Hold down a specific keyboard key until released
- key_up(key: str): Release a specific keyboard key
- press_keys(keys: Union[str, List[Union[str, List[str]]]]): Press keyboard keys (supports single keys, sequences, and combinations)

- take_screenshot(title_pattern: str = None, use_regex: bool = False, threshold: int = 10, scale_percent_for_ocr: int = None, save_to_downloads: bool = False, use_wgc: bool = False): Capture screen or window
- take_screenshot_with_ocr(title_pattern: str = None, use_regex: bool = False, threshold: int = 10, scale_percent_for_ocr: int = None, save_to_downloads: bool = False): Extract adn return text with coordinates using OCR from screen or window
- get_screen_size(): Get current screen resolution
- list_windows(): List all open windows
- activate_window(title_pattern: str, use_regex: bool = False, threshold: int = 60): Bring specified window to foreground
- wait_milliseconds(milliseconds: int): Wait for a specified number of milliseconds

You can quickly test the MCP server tools without an MCP client usingmcp-caller:

# List all available tools npx mcp-caller --server-json '{"type":"stdio","command":"uvx","args":["computer-control-mcp@latest","server"]}' --list-tools # Call a specific tool npx mcp-caller --server-json '{"type":"stdio","command":"uvx","args":["computer-control-mcp@latest","server"]}' --tool list_windows --params-json '{}'

# Clone the repository git clone https://github.com/AB498/computer-control-mcp.git cd computer-control-mcp # Build/Run: # 1. Install in development mode | Meaning that your edits to source code will be reflected in the installed package. pip install -e . # Then Start server | This is equivalent to uvx computer-control-mcp@latest just the local code is used computer-control-mcp # -- OR -- # 2. Build after pip install hatch | This needs version increment in orer to reflect code changes hatch build # Windows $latest = Get-ChildItem .\dist\.whl | Sort-Object LastWriteTime -Descending | Select-Object -First 1 pip install $latest.FullName --upgrade # Non-windows pip install dist/.whl --upgrade # Run computer-control-mcp

See theAPI Referencefor detailed information about the available functions and classes.

- Email (abcd49800@gmail.com)
- Discord (CodePlayground)

A local office automation assistant for file system operations, system command execution, and browser control.