DiffuGen

DiffuGen is a powerful MCP-based image generation system that brings cutting-edge AI models directly into your development workflow. It seamlessly integrates both Flux models (Flux Schnell, Flux Dev) and Stable Diffusion variants (SDXL, SD3, SD1.5) into a unified interface, allowing you to leverage the unique strengths of each model family without switching tools. With comprehensive parameter control and multi-GPU support, DiffuGen scales from rapid concept sketches on modest hardware to production-quality visuals on high-performance systems.

MCP (Model Context Protocol) is a protocol that enables LLMs (Large Language Models) to access custom tools and services. In simple terms, an MCP client (like Cursor, Windsurf, Roo Code, or Cline) can make requests to MCP servers to access tools that they provide.

DiffuGen functions as an MCP server that provides text-to-image generation capabilities. It implements the MCP protocol to allow compatible IDEs to send generation requests and receive generated images.
- generate_stable_diffusion_image: Generate with Stable Diffusion models
- generate_flux_image: Generate with Flux models

DiffuGen consists of several key components:

- setup-diffugen.sh: The complete install utility and model downloader and manager
- diffugen.py: The core Python script that implements the MCP server functionality and defines the generation tools
- diffugen.sh: A shell script launcher that sets up the environment and launches the Python server
- diffugen.json: Template configuration file for MCP integration with various IDEs (to be copied into IDE's MCP configuration)
- stable-diffusion.cpp: The optimized C++ implementation of Stable Diffusion used for actual image generation
- Receiving prompt and parameter data from an MCP client
- Processing the request through the Python server
- Calling the stable-diffusion.cpp binary with appropriate parameters
- Saving the generated image to a configured output directory
- Returning the path and metadata of the generated image to the client

- Significantly faster inference speed (up to 3-4x faster)
- Lower memory usage (works on GPUs with as little as 4GB VRAM)
- Optimized CUDA kernels for NVIDIA GPUs
- Support for various sampling methods and model formats
- Support for model quantization for better performance
- No Python dependencies for the core generation process

This allows DiffuGen to provide high-quality image generation with exceptional performance, even on modest hardware setups.

- Multiple Model Support: Generate images using various models including Flux Schnell, Flux Dev, SDXL, SD3, and SD1.5
- MCP Integration: Seamlessly integrates with IDEs that support MCP (Cursor, Windsurf, Roo Code, Cline, etc.)
- OpenAPI Server: Additional REST API interface for direct HTTP access to image generation capabilities
- Cross-Platform: Works on Linux, macOS, and Windows (via native or WSL)
- Parameter Control: Fine-tune your generations with controls for:

- Image dimensions (width/height)
- Sampling steps
- CFG scale
- Seed values
- Negative prompts (for SD models only, Flux does not support negative prompts.)
- Sampling methods

- CPU: 4-core processor (Intel i5/AMD Ryzen 5 or equivalent)
- RAM: 8GB system memory
- Storage: 5GB free disk space (SSD preferred for faster model loading)
- Python: 3.8 or newer
- GPU: Integrated graphics or entry-level dedicated GPU (optional)
- Network: Broadband connection for model downloads (5+ Mbps)

- CPU: 8+ core processor (Intel i7/i9 or AMD Ryzen 7/9)
- RAM: 16GB+ system memory
- GPU: NVIDIA GPU with 6GB+ VRAM (RTX 2060 or better for optimal performance)
- Storage: 20GB+ free SSD space
- Python: 3.10 or newer (3.11 offers best performance)
- Network: High-speed connection (20+ Mbps) for efficient model downloads

The easiest way to install DiffuGen is using the provided setup script:

git clone https://github.com/CLOUDWERX-DEV/diffugen.git cd DiffuGen chmod +x diffugen.sh chmod +x setup_diffugen.sh ./setup_diffugen.sh

Follow the interactive prompts to complete the installation.

- Install necessary dependencies
- Clone and build stable-diffusion.cpp
- Set up a Python virtual environment
- Download selected models (Note: Some models require Clip\VAE Models as well)
- Configure file paths for your system

If you prefer to install manually, follow these steps:

git clone https://github.com/CLOUDWERX-DEV/diffugen.git cd DiffuGen git clone --recursive https://github.com/leejet/stable-diffusion.cpp

cd stable-diffusion.cpp mkdir -p build && cd build

cmake .. -DCMAKE_BUILD_TYPE=Release -DSD_CUDA=ON make -j$(nproc) cd ../..

cmake .. -DCMAKE_BUILD_TYPE=Release make -j$(nproc) cd ../..

- Create and activate a Python virtual environment:

python3 -m venv diffugen_env source diffugen_env/bin/activate # On Windows: diffugen_env\Scripts\activate pip install -r requirements.txt

- Download required models (structure shown below):

stable-diffusion.cpp/models/ ├── ae.sft # VAE model ├── clip_l.safetensors # CLIP model ├── flux/ │ ├── flux1-schnell-q8_0.gguf # Flux Schnell model (default) │ └── flux1-dev-q8_0.gguf # Flux Dev model ├── sd3-medium.safetensors # SD3 model ├── sdxl-1.0-base.safetensors # SDXL model ├── sdxl_vae-fp16-fix.safetensors # SDXL VAE ├── t5xxl_fp16.safetensors # T5 model └── v1-5-pruned-emaonly.safetensors # SD1.5 model

You can download the models from the following sources:

# Create model directories mkdir -p stable-diffusion.cpp/models/flux # Flux models # Flux Schnell - Fast generation model (Q8 Quantized,requires t5xxl, clip-l, vae) curl -L https://huggingface.co/leejet/FLUX.1-schnell-gguf/resolve/main/flux1-schnell-q8_0.gguf -o stable-diffusion.cpp/models/flux/flux1-schnell-q8_0.gguf # Flux Dev - Development model with better quality (Q8 QUantized, requires t5xxl, clip-l, vae) curl -L https://huggingface.co/leejet/FLUX.1-dev-gguf/resolve/main/flux1-dev-q8_0.gguf -o stable-diffusion.cpp/models/flux/flux1-dev-q8_0.gguf # Required models for Flux # T5XXL Text Encoder curl -L https://huggingface.co/Sanami/flux1-dev-gguf/resolve/main/t5xxl_fp16.safetensors -o stable-diffusion.cpp/models/t5xxl_fp16.safetensors # CLIP-L Text Encoder curl -L https://huggingface.co/Sanami/flux1-dev-gguf/resolve/main/clip_l.safetensors -o stable-diffusion.cpp/models/clip_l.safetensors # VAE for image decoding curl -L https://huggingface.co/pretentioushorsefly/flux-models/resolve/main/models/vae/ae.safetensors -o stable-diffusion.cpp/models/ae.sft # Stable Diffusion models # SDXL 1.0 Base Model (requires sdxl-vae) curl -L https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors -o stable-diffusion.cpp/models/sd_xl_base_1.0.safetensors # SDXL VAE (required for SDXL) curl -L https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/sdxl_vae-fp16-fix.safetensors -o stable-diffusion.cpp/models/sdxl_vae-fp16-fix.safetensors # Stable Diffusion 1.5 (standalone) curl -L https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors -o stable-diffusion.cpp/models/v1-5-pruned-emaonly.safetensors # Stable Diffusion 3 Medium (standalone) curl -L https://huggingface.co/leo009/stable-diffusion-3-medium/resolve/main/sd3_medium_incl_clips_t5xxlfp16.safetensors -o stable-diffusion.cpp/models/sd3_medium_incl_clips_t5xxlfp16.safetensors

Note: Model download may take a long time depending on your internet connection. The SDXL model is approximately 6GB, SD3 is about 13GB, SD1.5 is around 4GB, and Flux models are 8-13GB each.

Configuration Approach: DiffuGen uses a single configuration file (diffugen.json) as the source of truth for all settings. The workflow is:
- Editdiffugen.jsonin the DiffuGen root directory with your desired settings
- Run option 5 insetup_diffugen.shto automatically update paths in this file
- Copy the content ofdiffugen.jsonto your IDE's MCP configuration file

The file contains all necessary settings:

- File paths (command, SD_CPP_PATH, models_dir, output_dir)
- Default model parameters (steps, cfg_scale, sampling_method)
- VRAM usage settings
- Metadata for IDE integration

Setting up with Claude in Anthropic Console

Claude can use DiffuGen if you've set it up as an MCP server on your system. When asking Claude to generate images, be specific about using DiffuGen and provide the parameters you want to use.

You should see:DiffuGen readywhen the server is successfully started.

The OpenAPI server provides a REST API interface for direct HTTP access to DiffuGen's image generation capabilities. This is in addition to the MCP integration and can be useful for:

- Direct HTTP API access
- Integration with other tools that don't support MCP
- Custom applications that need programmatic access

The server can be configured to use a different host or port if needed. By default, it runs on:

Generated images are saved to the/outputdirectory by default. If this directory is not accessible, the server will automatically create anoutputdirectory in the current working directory. Images are served through the/imagesendpoint.
- Open OpenWebUI Settings (gear icon)
- Navigate to the "Tools" section
- Click the "+" button to add a new tool server
- Enter the following details:

Once added, DiffuGen will appear in the available tools list when clicking the tools icon in the chat interface. The following endpoints will be available:

- generate_stable_image_generate_stable_post: Generate with Stable Diffusion
- generate_flux_image_endpoint_generate_flux_post: Generate with Flux Models
- list_models_models_get: List Available Models

curl -X POST "http://0.0.0.0:5199/generate/flux" \ -H "Content-Type: application/json" \ -d '{"prompt": "A beautiful sunset", "model": "flux-schnell"}'

import requests response = requests.post( "http://0.0.0.0:5199/generate/flux", json={ "prompt": "A beautiful sunset", "model": "flux-schnell" } ) result = response.json()

Each model has specific default parameters optimized for best results:

These default parameters can be customized by adding adefault_paramssection to your IDE's MCP configuration file:

"default_params": { "steps": { "flux-schnell": 12, // Customize steps for better quality "sdxl": 30 // Increase steps for more detailed SDXL images }, "cfg_scale": { "sd15": 9.0 // Higher cfg_scale for stronger prompt adherence } }

You only need to specify the parameters you want to override - any unspecified values will use the built-in defaults.

Here are examples of how to ask an AI assistant to generate images with DiffuGen:

Generate an image of a cat playing with yarn

Create a picture of a futuristic cityscape with flying cars

Generate an image of a medieval castle using the sdxl model

Create a flux image of a sunset over mountains

Generate an image of a cyberpunk street scene, model=flux-dev, width=768, height=512, steps=25, cfg_scale=1.0, seed=42

Create an illustration of a fantasy character with model=sd15, width=512, height=768, steps=30, cfg_scale=7.5, sampling_method=dpm++2m, negative_prompt=blurry, low quality, distorted

DiffuGen can be used from the command line with the following basic syntax: