MCP YOLOE: Zero-Shot Object Detection & Segmentation

by rjn32s

278 downloads Not rated yet

GitHub

Other
AI

About

Provide your AI agents with "eyes." This server enables open-vocabulary object detection and instance segmentation using naturally phrased text prompts (e.g., "detect the laptop next to the coffee").

Explore

- Zero-Shot Detection: Detect arbitrary objects using natural language prompts.
- Precision Segmentation: Get exact polygon masks for every detected object.
- Flexible Inputs: Works with local file paths, remote image URLs, and Base64 strings.
- Agent-First: Designed specifically for integration with Claude, IDEs, and autonomous workspace agents.

Setting up with Highlight

This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:

Download and install Highlight from highlightai.com/download
Navigate to the plugins tab and select "Add Custom Plugin"
Configure the plugin with the settings below

Plugin Name MCP YOLOE: Zero-Shot Object Detection & Segmentation

Command (node, npx, python, etc.)

Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
Enable "Start Automatically" if you want the plugin to start when Highlight launches

Setting up with Cursor

Open Cursor's settings
Navigate to the MCP section
Click "Add new Global MCP server"
Copy and paste the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

Setting up with Claude Desktop

Open Claude Desktop's settings
Go to the Integrations tab
Click "Add MCP Server"
Use the configuration below based on your operating system

Note: Make sure Claude Desktop is running before attempting to connect.

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

Setting up with Windsurf

Open Windsurf's preferences
Go to the Integrations section
Click "Add MCP Integration"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

Setting up with Cline

Open Cline's settings
Navigate to the Integrations section
Click "Add MCP Server"
Copy and paste the configuration below based on your operating system

Note: Make sure to restart Cline after adding the configuration.

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

Setting up with Goose

Open Goose's preferences
Go to the Integrations tab
Select "MCP Servers"
Click "Add New Server"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

From the repository

Ask your agent to:
> "Find the 'vintage typewriter' in this image and give me its exact coordinates."

Claude Desktop / Cursor

Paste into your MCP client config file to install this server.

{
    "mcpServers": {
        "mcp yoloe: zero-shot object detection & segmentation": {
            "mcp-yolo": {
                "command": "uvx",
                "args": [
                    "mcp-yolo"
                ]
            }
        }
    }
}

McpServers

{
    "mcp-yolo": {
        "command": "uvx",
        "args": [
            "mcp-yolo"
        ]
    }
}

MCP-YOLO is a powerful Model Context Protocol server that grants AI agents advanced computer vision capabilities. Unlike traditional YOLO models that only detect a fixed list of objects, this server uses Zero-Shot Learning to detect and segment anything you describe.

Key Features

- Zero-Shot Detection: Detect arbitrary objects using natural language prompts. - Precision Segmentation: Get exact polygon masks for every detected object. - Flexible Inputs: Works with local file paths, remote image URLs, and Base64 strings. - Agent-First: Designed specifically for integration with Claude, IDEs, and autonomous workspace agents.

Example Usage

Ask your agent to: > "Find the 'vintage typewriter' in this image and give me its exact coordinates."

Performance

Uses the state-of-the-art YOLOE26-L architecture, providing a perfect balance of high precision (55.0 mAP) and rapid inference (~6.2ms on T4 GPUs).

No reviews yet — be the first

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.

About

Explore

Setting up with Highlight

From the repository

Claude Desktop / Cursor

McpServers

Key Features

Example Usage

Performance

More like this

Sequential Thinking

Filesystem

EverArt

Google Drive