MCP QEMU VM Control

Give your AI full computer access — safely. Let Claude (or any MCP-compatible LLM) see your screen, move the mouse, type on the keyboard, and run commands — all inside an isolated QEMU virtual machine. Perfect for AI-driven automation, testing, and computer-use experiments without risking your host system.

Give your AI full computer access — safely.

Let Claude (or any MCP-compatible LLM) see your screen, move the mouse, type on the keyboard, and run commands — all inside an isolated QEMU virtual machine. Perfect for AI-driven automation, testing, and computer-use experiments without risking your host system.

A Model Context Protocol (MCP) server for controlling QEMU virtual machines via SSH. This server enables LLMs to interact with VMs through mouse/keyboard control, screenshots, and SSH command execution.

- Features
- Prerequisites
- QEMU/libvirt Setup
- Installation
- Configuration
- Usage
- Tools Reference
- Typical Workflow
- Best Practices for LLM Automation
- Architecture
- Known Issues & Limitations
- Troubleshooting

sudo pacman -S qemu-full libvirt virt-manager dnsmasq iptables-nft

sudo apt install qemu-kvm libvirt-daemon-system libvirt-clients virt-manager bridge-utils

# Enable and start libvirtd sudo systemctl enable --now libvirtd # Add your user to libvirt group sudo usermod -aG libvirt $USER # Log out and back in, then verify groups # should show 'libvirt'

# Check network status virsh -c qemu:///system net-list --all # If 'default' is not active, start it virsh -c qemu:///system net-start default # Enable autostart virsh -c qemu:///system net-autostart default

# Inside the VM - Install required packages # Arch/Manjaro sudo pacman -S --needed openssh xdotool scrot xorg-xrandr xorg-xinput # Debian/Ubuntu sudo apt install openssh-server xdotool scrot x11-xserver-utils xinput # Enable SSH sudo systemctl enable --now sshd

# Create vmrobot user sudo useradd -m -s /bin/bash vmrobot sudo passwd vmrobot # Set up SSH key authentication sudo -u vmrobot mkdir -p /home/vmrobot/.ssh sudo -u vmrobot chmod 700 /home/vmrobot/.ssh

# Copy your public key to the VM ssh-copy-id vmrobot@192.168.122.XX # Or manually add to /home/vmrobot/.ssh/authorized_keys on VM

# Quick fix (run once per session) xhost +local:vmrobot # Permanent fix - add to ~/.xprofile or ~/.xinitrc echo "xhost +local:" >> ~/.xprofile

# On the VM, allow vmrobot to run commands as your desktop user echo 'vmrobot ALL=(sergey) NOPASSWD: ALL' | sudo tee /etc/sudoers.d/vmrobot-desktop sudo chmod 440 /etc/sudoers.d/vmrobot-desktop

# From the host virsh -c qemu:///system domifaddr manjaro # Or from inside the VM ip addr show | grep "inet 192.168.122"

# Test SSH ssh vmrobot@192.168.122.XX # Test X11 automation ssh vmrobot@192.168.122.XX 'DISPLAY=:0 xdotool getmouselocation' # Test screenshot ssh vmrobot@192.168.122.XX 'DISPLAY=:0 scrot /tmp/test.png && echo Success'

git clone https://github.com/Neanderthal/mcp-qemu-vm.git cd mcp-qemu-vm

uv venv && source .venv/bin/activate uv pip install -r requirements.txt

python -m venv .venv source .venv/bin/activate pip install -r requirements.txt

{ "qemu-vm-control": { "command": "python3", "args": ["/path/to/mcp-qemu-vm/server.py"], "env": { "VM_HOST": "192.168.122.79", "VM_USER": "vmrobot", "VM_PORT": "22", "VM_DISPLAY": ":0" } } }

- macOS:~/Library/Application Support/Claude/claude_desktop_config.json
- Windows:%APPDATA%/Claude/claude_desktop_config.json
- Linux:~/.config/Claude/claude_desktop_config.json

uv run mcp dev server.py # With custom environment VM_HOST=192.168.122.79 VM_USER=vmrobot uv run mcp dev server.py

Projects organize all outputs (screenshots, logs, results, advice) into timestamped folders underdata/projects/.

[ {"action": "press_keys", "keys": ["Ctrl", "Shift", "p"]}, {"action": "wait", "seconds": 0.5}, {"action": "type_text", "text": "Terminal: Focus Terminal"}, {"action": "press_keys", "keys": ["Return"]} ]

Locate on-screen elements by their visible text — exact pixel coordinates, no coordinate guessing. The host OCRs the full-resolution screenshot (tesseract) and maps the match straight into the click path. Works on any visible text, including nested Citrix/web where accessibility APIs can't reach; does not find unlabeled icons.

click_text("Submit") # finds "Submit" and clicks its exact center find_text("File") # lists all matches with coordinates click_text("OK", index=1) # click the 2nd "OK" if several match

Zoom (magnify a region, then click it precisely)

When detail is too small/low-contrast to resolve in the full screenshot, magnify a region and click within it. The serverkeeps the crop mapping, so a point you pick in the zoomed image maps back to the exact full-screen pixel — no coordinate math.

zoom(800, 600, width=400, height=300, scale=4) # view a 4× magnified crop click_zoomed(610, 250) # click that spot → exact full-screen pixel

For dense or ambiguous screens, overlaynumbered markson every detected text element and pick one by its number — a discrete choice that's far more reliable than estimating coordinates.

mark_screen() # view the annotated screenshot + legend (0 -> "File", 1 -> "Edit", …) click_mark(1) # click element #1 at its exact center

Host requirements:tesseract(the binary) pluspillowandpytesseractin the server's Python env. These are optional — the rest of the server runs without them; only the OCR (find_text/click_text) and zoom (zoom) tools need them (zoomneeds onlypillow):

# Arch/Manjaro host sudo pacman -S tesseract tesseract-data-eng uv pip install pillow pytesseract

Screenshots are saved to the project'sscreenshots/folder and exposed as MCP resources atvm://screenshot/{id}.

Scale factors are auto-detected at startup (HiDPI/scaling mismatches); coordinate tools apply them transparently.

1. project_init("my-task", "Description") 2. take_screenshot() 3. ... perform VM operations ... 4. project_read_logs() 5. project_save_result("output.txt", data) 6. project_save_advice("Title", "Lessons learned...")

1. project_list() 2. project_load("data/projects/...") # Shows any saved advice 3. ... continue work ...

These lessons were learned from real-world usage and help avoid common pitfalls.
- take_screenshot()
- Analyze the image
- Identify current focus (which window/field is active)
- Only then proceed with actions

Never skip screenshots to "save time"- blind actions lead to errors.

2. Don't Trust Mouse Clicks for Focus

Clicking on a window/terminal does NOT reliably switch focus, especially in:

- Nested environments (Citrix, remote desktop)
- High-latency connections
- Applications with multiple panels (VS Code, IDEs)

[ {"action": "press_keys", "keys": ["Ctrl", "Shift", "p"]}, {"action": "wait", "seconds": 0.5}, {"action": "type_text", "text": "Terminal: Focus Terminal"}, {"action": "wait", "seconds": 0.3}, {"action": "press_keys", "keys": ["Return"]}, {"action": "wait", "seconds": 0.5} ]

Thentake_screenshot()to verify before typing.

Never rapid-fire actions- they may arrive out of order.

Userun_actions()instead of separate tool calls to reduce latency and ensure ordering:

# Instead of 5 separate calls: run_actions([ {"action": "press_keys", "keys": ["Ctrl", "Shift", "p"]}, {"action": "wait", "seconds": 0.5}, {"action": "type_text", "text": "command"}, {"action": "wait", "seconds": 0.3}, {"action": "press_keys", "keys": ["Return"]} ])

ssh_executeonly reaches thefirst VM layer. For nested environments (VM → Citrix → Windows), use UI automation to type commands in the visible terminal.
- Typing immediately after clicking terminal (focus may not have switched)
- Skipping screenshots to "save time"
- Usingssh_executefor nested environment commands
- Not waiting between actions
- Assuming focus switched without verification

┌─────────────┐ SSH ┌──────────────┐ │ │ ◄──────────────────► │ │ │ MCP Server │ │ QEMU VM │ │ (Host) │ │ (Linux) │ │ │ │ │ └──────┬──────┘ └──────────────┘ │ │ │ MCP Protocol │ │ (stdio) │ │ │ ▼ ▼ ┌─────────────┐ xdotool, scrot │ LLM Client │ X11 automation │ (Claude) │ └─────────────┘

┌────────────────────────────────────────────────────┐ │ Host (192.168.122.1) │ │ ┌──────────┐ │ │ │ virbr0 │◄── NAT bridge │ │ └────┬─────┘ │ │ │ │ │ ┌────┴─────┐ │ │ │ QEMU VM │ 192.168.122.79 │ │ │ (manjaro)│ │ │ └──────────┘ │ └────────────────────────────────────────────────────┘

All xdotool interactions are built from a small set of purecommand builders(_type_cmd,_keys_cmd,_click_cmd,_move_cmd) so the shell command for an action is constructed in exactly one place. Each builder takes an alreadyshlex.quote()d display string and returns the command to run on the VM; the builders also own input validation (key-name pattern, button map, click-count clamp) and the UTF-8 locale prefix for typing.

- Standalone tools(move_mouse,click,type_text,press_keys,wait) — individually exposed MCP tools with typed signatures and rich docstrings.
- run_actions— the batch path. It dispatches throughACTION_HANDLERS, a{name: async handler}registry that is thesingle source of truthfor which actions a batch supports. Each handler shares the signatureasync (app_ctx, display, action_dict) -> summary. Unknown action names raise and stop the batch (consistent with its "stops on first error" contract).

run_actions(actions) │ for each action ▼ ACTION_HANDLERS[name] ──► _act_(app, display, action) │ uses ▼ _type_cmd / _keys_cmd / _click_cmd / _move_cmd │ ▼ run_vm_cmd(ssh, …) ──► xdotool over SSH

Adding a new batch action:write a_act_<name>(app, display, action)handler (reusing or adding a__cmdbuilder) and add one entry toACTION_HANDLERS. No changes to the dispatch loop are needed.

mcp-qemu-vm/ ├── server.py # Main MCP server (single file) ├── pyproject.toml # Project metadata, ruff & pytest config ├── requirements.txt # Python dependencies ├── .env.example # Documented env var template ├── test_ssh_tools.py # Unit tests (no-VM) + manual SSH smoke check ├── LICENSE # MIT ├── data/ │ └── projects/ # Project folders │ └── YYYYMMDD-HHMMSS_name/ │ ├── screenshots/ │ ├── logs/ │ ├── results/ │ └── advice/ └── README.md

Issues confirmed in real nested-environment use (host → Citrix → Windows → Outlook). Each lists the symptom, the root cause, and the current workaround.

#1 and #2 are fixed inserver.py.#3–#6 are inherent limitations of the nested environment (Citrix/RDP session policy) or the architecture (SSH lands on the first VM layer only) — they can't be fixed in this server, so the workarounds remain the recommended approach.

1.type_textfails on Cyrillic / non-ASCII text — FIXED

- Symptom:type_text(and anyxdotool typewith non-ASCII) errors out with exit status 1. Direct run reveals:Invalid multi-byte sequence encountered / xdo_enter_text_window reported an error. ASCII text types fine.
- Root cause:xdotool typedecodes multi-byte input using the current locale, but thevmrobot/ desktop-user SSH environment hasno UTF-8 locale(LANGempty, keyboard layout bareus). Without a UTF-8LC_CTYPE, multi-byte UTF-8 (Cyrillic, etc.) cannot be decoded.
- Fix (applied):type_textand therun_actionstype step now prefix the xdotool invocation withLC_ALL=$VM_LOCALE(defaultC.UTF-8), so non-ASCII text works out of the box. Override with theVM_LOCALEenv var if the VM lacksC.UTF-8(e.g. setVM_LOCALE=ru_RU.utf8; check available locales withlocale -a).

2. Embedded newlines in typed text become literal glyphs, not Enter — FIXED

- Symptom:Typing multi-line text (e.g.xdotool typewith\n, ortype --file -) into a rich editor like Outlook produces onerun-on paragraphwith stray box/control-character glyphs where the line breaks should be — paragraph breaks are lost.
- Root cause:In this nested Citrix → Windows path, the\n(LF) is delivered as a literal control character to the editor instead of being interpreted as a Return keypress.
- Fix (applied):type_text(and therun_actionstype step) now split text on newlines, type each line via stdin, and send line breaks as explicitReturnkey presses instead of a literal LF.\r\nand\rare normalised first. This works in both terminals and rich editors — no caller-side splitting needed.

3. Clipboard redirection may be disabled in the guest session

- Symptom:Setting the host/X clipboard (xclip -selection clipboard) and pasting withCtrl+Vdoesnottransfer text into the Windows/Citrix layer.
- Root cause:Clipboard redirection is turned off in the Citrix/RDP session policy, so the inner session has its own isolated clipboard.
- Workaround:do not rely on copy/paste to inject text across the nesting boundary; fall back to typing (see issues #1 and #2).