FunASR-Powered MCP Server (MCPServer)

Overview

MCPServer is a Python-based server that leverages Alibaba's FunASR library to provide speech processing services through the FastMCP framework. It offers tools for:

- Audio Validation: Checking if audio files are valid and readable, and providing their properties.
- Speech Transcription: Asynchronously transcribing speech from audio files using advanced ASR models like Paraformer. Supports managing transcription tasks and retrieving results, including detailed timestamp information.
- Voice Activity Detection (VAD): Identifying speech segments in audio files.

The server is designed to be extensible and allows for dynamic loading and switching of ASR and VAD models.

Features

- Audio File Validation: Verifies audio file integrity, readability, and format.
- Asynchronous Speech-to-Text Transcription: Non-blocking transcription suitable for long audio files.
- Transcription Task Management: Start tasks, query status, and retrieve results.
- Detailed Transcription Results: Access to full transcription text, segment-level start/end times, and word-level timestamps (if provided by the ASR model).
- Voice Activity Detection (VAD): Returns precise start and end timestamps of speech segments in an audio file.
- Multi-Model Support: Leverages FunASR's diverse model zoo for both ASR and VAD.
- Dynamic Model Configuration:
- Specify models per transcription or VAD request.
- Explicitly load/switch the default ASR and VAD models used by the server instance.
- Configurable Model Parameters: Pass specific loading and generation arguments to FunASR models.

Prerequisites

- Python: 3.8+
- Pip: For installing Python packages.
- MODELSCOPE_API_TOKEN (Optional):
- FunASR downloads models from ModelScope. If you encounter rate limits or need to access private models, you might need to set the MODELSCOPE_API_TOKEN environment variable.
- You can obtain a token from the ModelScope website.
- Set it in your environment: export MODELSCOPE_API_TOKEN="YOUR_TOKEN_HERE"

Setup and Installation

1. Clone the Repository (if applicable):
If this server is part of a larger repository, clone it. Otherwise, ensure you have the MCPServer directory and its contents.

2. Create and Activate a Virtual Environment (Recommended):

    python -m venv .venv

    source .venv/bin/activate  # On Windows: .venv\Scripts\activate

    

3. Install Dependencies:
Navigate to the MCPServer directory (the one containing this README and server.py).
Install the required packages:

    pip install -r requirements.txt

    

This will install fastmcp, funasr, and their dependencies, including PyTorch (CPU version by default if not specified otherwise by FunASR's direct dependencies). If you have specific PyTorch needs (e.g., a GPU version), it's recommended to install PyTorch manually before running the command above, following instructions from the official PyTorch website.

Running the Server

1. Navigate to the MCPServer directory.
2. Run the server application:

    uvicorn main:app --host 0.0.0.0 --port 9000

    

3. The server will start, and you should see log output indicating it's running, typically on http://0.0.0.0:9000. On the first run, FunASR will download the default ASR and VAD models, which may take some time.

Available MCP Tools

> MCPServer: http://0.0.0.0:9000/sse

You can interact with these tools using any MCP client (e.g., mcp_client or via HTTP requests). The server provides the following tools:

---

1. validate_audio_file

- Description: Validates an audio file to check if it's suitable for processing and provides its properties. - Parameters: - file_path (str, required): Path to the audio file. - Example Return (Success):

     {
       "status": "valid",
       "message": "Audio file is valid.",
       "details": {
         "samplerate": 16000,
         "channels": 1,
         "duration": 10.5,
         "formatted_duration": "00:10.500",
         "format": "WAV",
         "subtype": "PCM_16"
       }
     }
     

- Example Return (Error - File Not Found):

     {
        "status": "invalid",
        "message": "Error: File not found at 'path/to/non_existent_audio.wav'.",
        "details": null
     }
     

---

2. start_speech_transcription

- Description: Starts an asynchronous speech transcription task for the given audio file. Allows specifying ASR model and generation parameters. - Parameters: - audio_path (str, required): Path to the audio file. - model_name (str, optional): Specific ASR model to use for this task (e.g., a ModelScope ID). Overrides the server's current default ASR model. If the specified model is not already loaded with compatible settings, the server will attempt to load it using its default load parameters for that model or the instance's general default load parameters. - model_generate_kwargs (dict, optional): Specific arguments for the ASR model's generate method (e.g., {"batch_size_s": 60, "hotword": "特定热词"}). These override any default generation arguments set in the server for the current ASR model. - Example Return (Success):

     {
       "task_id": "a1b2c3d4-e5f6-7890-1234-567890abcdef",
       "status": "processing_started",
       "message": "Transcription task started and is now processing."
     }
     

- Example Return (Error - Invalid Audio):

     {
        "task_id": null,
        "status": "error",
        "message": "Error: File at '/path/to/your/bad_audio.wav' is not a valid audio file or is corrupted. Details: <error from soundfile>",
        "details": null
     }
     

- Example Return (Error - Model Load Failure during task):

     {
        "task_id": null,
        "status": "error",
        "message": "Failed to switch to model 'non_existent_model_id'. Error: Error loading model 'non_existent_model_id': <actual error>"
     }
     

---

3. get_transcription_task_status

- Description: Queries the status of a previously started speech transcription task. - Parameters: - task_id (str, required): The unique ID of the transcription task. - Example Return (Processing):

     {
        "status": "processing",
        "audio_path": "/path/to/your/audio.wav",
        "model_used": "iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch",
        "submitted_at": "YYYY-MM-DDTHH:MM:SS.ffffff+00:00",
        "details_from_validation": { / ... audio details from validate_audio ... / },
        "model_generate_kwargs": {"batch_size_s": 300, "hotword": "魔搭"},
        "processing_started_at": "YYYY-MM-DDTHH:MM:SS.ffffff+00:00"
     }
     

- Example Return (Completed):

     {
        "status": "completed",
        "audio_path": "/path/to/your/audio.wav",
        "model_used": "iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch",
        "submitted_at": "YYYY-MM-DDTHH:MM:SS.ffffff+00:00",
        "details_from_validation": { / ... / },
        "model_generate_kwargs": { / ... / },
        "processing_started_at": "YYYY-MM-DDTHH:MM:SS.ffffff+00:00",
        "result": [ / ... actual transcription result ... / ],
        "completed_at": "YYYY-MM-DDTHH:MM:SS.ffffff+00:00"
     }
     

- Example Return (Error - Task Not Found):

      {
          "status": "error",
          "message": "Task ID not found."
      }
      

---

4. get_transcription_result

- Description: Retrieves the result of a completed speech transcription task. - Parameters: - task_id (str, required): The unique ID of the transcription task. - Example Return (Success/Completed):

     {
       "task_id": "a1b2c3d4-e5f6-7890-1234-567890abcdef",
       "status": "completed",
       "result": [
         {
           "text": "这是 一段 测试 文本",
           "start": 120,
           "end": 2850,
           "timestamp": [[120, 300], [330, 500], [550, 900], [920, 1200]]
         }
       ],
       "completed_at": "YYYY-MM-DDTHH:MM:SS.ffffff+00:00"
     }
     

- Example Return (Task Still Processing):

     {
        "task_id": "a1b2c3d4-e5f6-7890-1234-567890abcdef",
        "status": "processing",
        "message": "Transcription not yet completed or has failed."
     }
     

- Example Return (Task Failed):

     {
        "task_id": "a1b2c3d4-e5f6-7890-1234-567890abcdef",
        "status": "failed",
        "message": "Transcription failed.",
        "error_details": "Description of the error during transcription.",
        "failed_at": "YYYY-MM-DDTHH:MM:SS.ffffff+00:00"
     }
     

---

5. load_asr_model

- Description: Loads or reloads a specific ASR model, making it the default for subsequent tasks unless overridden. Returns status of operation. - Parameters: - model_name (str, required): The FunASR model identifier to load (e.g., a ModelScope ID). - device (str, optional): Device to load the model on (e.g., "cpu", "cuda:0"). Uses instance default if None. - model_load_kwargs (dict, optional): Specific arguments for loading the ASR model (e.g., {"ncpu": 2, "vad_model": "other-vad-id", "punc_model": "other-punc-id"}). These will be passed to funasr.AutoModel. - Example Return (Success):

     {
       "status": "success",
       "message": "Model 'iic/speech_paraformer-large-en-16k-common-vocab10020' loaded successfully on cpu with load_kwargs: {'ncpu': 2, 'vad_model': 'fsmn-vad'}."
     }
     

- Example Return (Error):

     {
        "status": "error",
        "message": "Error loading model 'invalid-model-id': <FunASR or ModelScope error details>"
     }
     

---

6. get_voice_activity_segments

- Description: Detects speech segments in an audio file using a Voice Activity Detection (VAD) model. - Parameters: - audio_path (str, required): Path to the audio file. - vad_model_name (str, optional): Specific VAD model to use. Overrides the server's current default VAD model. - model_load_kwargs (dict, optional): Specific arguments for loading the VAD model if vad_model_name is specified and different from the currently loaded one. - model_generate_kwargs (dict, optional): Specific arguments for the VAD model's generate method. - Example Return (Success):

     {
       "status": "success",
       "segments": [ [100, 2500], [3000, 5500] ],
       "audio_path": "path/to/your/audio.wav",
       "vad_model_used": "damo/speech_fsmn_vad_zh-cn-16k-common-pytorch",
       "generate_kwargs_used": {},
       "audio_details": { / ... audio properties ... / }
     }
     

- Example Return (Error - VAD Processing Failed):

     {
        "status": "error",
        "message": "VAD processing failed for 'path/to/audio.wav': <FunASR error details>",
        "audio_path": "path/to/audio.wav",
        "vad_model_used": "damo/speech_fsmn_vad_zh-cn-16k-common-pytorch"
     }
     

---

7. load_vad_model

- Description: Loads or reloads a specific VAD model, making it the default for subsequent VAD tasks unless overridden. Returns status of operation. - Parameters: - model_name (str, required): The FunASR VAD model identifier to load. - device (str, optional): Device to load the model on. Uses instance default if None. - ncpu (int, optional): Number of CPU threads if device is CPU. Uses instance default if None. - model_load_kwargs (dict, optional): Specific arguments for loading the VAD model. - Example Return (Success):

     {
       "status": "success",
       "message": "VAD Model 'damo/speech_fsmn_vad_zh-cn-16k-common-pytorch' loaded successfully on cpu with load_kwargs: {'ncpu': 2}."
     }
     

---

Example Usage with curl

Here's how you might call the start_speech_transcription tool using curl:

```bash