Computer Use AI SDK

by m13v

195 313 downloads Not rated yet

GitHub

Developer Tools

About

# Computer Use AI SDK * We've built an MCP server that controls computer * You've heard of OpenAI's operator, you've heard of Claude's computer use. Now the open source alternative: Computer Use SDK from screenpipe. * It's native on macOS—no virtual machine bs, no guardrails. Use it with any app or website however you…

Explore

- Launches apps, reads content, clicks, enters text, and presses keys.
- Uses desktop‑rendered elements for speed and reliability.
- Native on macOS with no virtual machine overhead.
- No guardrails — full control over any app or website.
- Provides both CLI and web‑based client interfaces.
- Integrates with Anthropic’s API for AI agent capabilities.

Setting up with Highlight

This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:

Download and install Highlight from highlightai.com/download
Navigate to the plugins tab and select "Add Custom Plugin"
Configure the plugin with the settings below

Plugin Name Computer Use AI SDK

Command (node, npx, python, etc.)

Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
Enable "Start Automatically" if you want the plugin to start when Highlight launches

Setting up with Cursor

Open Cursor's settings
Navigate to the MCP section
Click "Add new Global MCP server"
Copy and paste the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

Setting up with Claude Desktop

Open Claude Desktop's settings
Go to the Integrations tab
Click "Add MCP Server"
Use the configuration below based on your operating system

Note: Make sure Claude Desktop is running before attempting to connect.

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

Setting up with Windsurf

Open Windsurf's preferences
Go to the Integrations section
Click "Add MCP Integration"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

Setting up with Cline

Open Cline's settings
Navigate to the Integrations section
Click "Add MCP Server"
Copy and paste the configuration below based on your operating system

Note: Make sure to restart Cline after adding the configuration.

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

Setting up with Goose

Open Goose's preferences
Go to the Integrations tab
Select "MCP Servers"
Click "Add New Server"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

From the repository

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

export ANTHROPIC_API_KEY=sk-ant-xxxx # Replace with your actual Anthropic API key

npx tsx main.ts

Claude Desktop / Cursor

Paste into your MCP client config file to install this server.

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

McpServers

{
    "MCP-server-client-computer-use-ai-sdk": {
        "command": "npx",
        "args": [
            "tsx",
            "main.ts"
        ]
    }
}

We've built an MCP server that controls computer

You've heard of OpenAI's operator, you've heard of Claude's computer use. Now the open source alternative: Computer Use SDK from screenpipe.

It's native on macOS—no virtual machine bs, no guardrails. Use it with any app or website however you want.

No pixel-based bs—it relies on underlying desktop-rendered elements, making it much faster and far more reliable than pixel-based vision models.

You can now build your own agents getting started with our simple Hello World Template using our MCP server and client.

There are tools that our MCP Server provides out of the box:
Launch apps
Read content
Click
Enter text
Press keys

These will be computational primitives to allow the AI to control your computer and do your tasks for you. What will you build? Come check us out at https://screenpi.pe

Demos

agent sending a message

https://github.com/user-attachments/assets/f8687500-9a8c-4a96-81b6-77562feff093

get latest whatsapp messages

open arc browser

Get started

git clone https://github.com/m13v/computer-use-ai-sdk.git
cd MCP-server-client-computer-use-ai-sdk

```bash

No reviews yet — be the first

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.

About

Explore

Setting up with Highlight

From the repository

Claude Desktop / Cursor

McpServers

Demos

Get started

More like this

Sentry

Github

Git

GitLab