GitHub Models Helper MCP Server
About
# GitHub Models Helper MCP Server This MCP server helps you interact with and compare different language models available via GitHub Models and AzureML, including OpenAI, Microsoft, Meta, Mistral, and more. ## Features - List available language models with metadata - Compare responses from different models for the…
Explore
- List available language models with metadata
- Compare responses from different models for the same prompt
- Filter and sort models by various criteria
- Comprehensive error handling and fallbacks
- Visualize model comparisons (see below)
Setting up with Highlight
This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:
- Download and install Highlight from highlightai.com/download
- Navigate to the plugins tab and select "Add Custom Plugin"
-
Configure the plugin with the settings below
Plugin Name
GitHub Models Helper MCP ServerCommand (node, npx, python, etc.)Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
- Enable "Start Automatically" if you want the plugin to start when Highlight launches
From the repository
1. Install dependencies:
npm install
2. Set up environment variables:
Copy
.env.template to .env and add your GitHub token: GITHUB_TOKEN=your_github_personal_access_token
3. Build the project:
npm run build
4. Run the MCP server in development mode:
npx @modelcontextprotocol/inspector dist/index.js
5. Add the MCP server to Claude Desktop:
In
claude_desktop_config.json: {
"mcpServers": {
"GitHub Models Helper": {
"command": "node",
"args": [
"/absolute/path/to/gh-models-helper/dist/index.js"
],
"env": {
"GITHUB_TOKEN": "your_github_personal_access_token"
}
}
}
}
Claude Desktop / Cursor
Paste into your MCP client config file to install this server.
{
"mcpServers": {
"github models helper mcp server": {
"mcp-server-gh-models-helper": {
"command": "npx",
"args": [
"@modelcontextprotocol/inspector",
"dist/index.js"
]
}
}
}
}McpServers
{
"mcp-server-gh-models-helper": {
"command": "npx",
"args": [
"@modelcontextprotocol/inspector",
"dist/index.js"
]
}
}This MCP server helps you interact with and compare different language models available via GitHub Models and AzureML, including OpenAI, Microsoft, Meta, Mistral, and more.
Features
- List available language models with metadata
- Compare responses from different models for the same prompt
- Filter and sort models by various criteria
- Comprehensive error handling and fallbacks
- Visualize model comparisons (see below)
Getting Started
1. Install dependencies:
npm install
2. Set up environment variables:
Copy
.env.template to .env and add your GitHub token: GITHUB_TOKEN=your_github_personal_access_token
3. Build the project:
npm run build
4. Run the MCP server in development mode:
npx @modelcontextprotocol/inspector dist/index.js
5. Add the MCP server to Claude Desktop:
In
claude_desktop_config.json: {
"mcpServers": {
"GitHub Models Helper": {
"command": "node",
"args": [
"/absolute/path/to/gh-models-helper/dist/index.js"
],
"env": {
"GITHUB_TOKEN": "your_github_personal_access_token"
}
}
}
}
Available Phi-3 Models
| Model ID | Display Name | Context Window | Summary |
|------------------------------|-------------------------------|---------------|--------------------------------------------------------------|
| Phi-3-medium-128k-instruct | Phi-3-medium instruct (128k) | 131,072 | Same Phi-3-medium model, but with a larger context size |
| Phi-3-medium-4k-instruct | Phi-3-medium instruct (4k) | 4,096 | 14B parameters, better quality than Phi-3-mini |
> Note: There is currently no model named "Phi-4" or "Phi-3-mini-4k-instruct" in the available list. Use the above IDs for comparisons.
Visualizing Model Comparisons
You can compare how different models respond to the same prompt and visualize the results. For example, to compare three models:
- Phi-3-medium-128k-instruct
- gpt-4o-mini
- Mistral-small
Example prompt:
Explain the difference between AI and machine learning.
Sample output visualization:
| Model | Response |
|------------------------------|----------|
| Phi-3-medium-128k-instruct | ... |
| gpt-4o-mini | ... |
| Mistral-small | ... |
You can use your own prompt and models. The server will return a JSON object with the responses, which you can render as a table or chart in your application.
Example Prompts
- "list all available phi-3 models"
- "compare Phi-3-medium-4k-instruct and Mistral-small on this prompt: how many ns in bananasss??"
- "Do a comparison between the Phi-3-medium-128k-instruct, gpt-4o-mini, and Mistral-small models"
---
For more details, see the code and documentation in project.md.
Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
