Search Engine with RAG and MCP
About
Search Engine with RAG and MCP is a search engine that combines LangChain, Model Context Protocol (MCP), Retrieval-Augmented Generation (RAG), and Ollama to create an agentic AI system capable of searching the web, retrieving information, and providing relevant answers.
Details
- License
- MIT
Explore
- Web search capabilities using the Exa API
- Web content retrieval using FireCrawl
- RAG (Retrieval-Augmented Generation) for more relevant information extraction
- MCP (Model Context Protocol) server for standardized tool invocation
- Support for both local LLMs via Ollama and cloud-based LLMs via OpenAI
- Flexible architecture supporting direct search, agent-based search, or server mode
- Comprehensive error handling and graceful fallbacks
- Python 3.13+ with type hints
- Asynchronous processing for efficient web operations
Setting up with Highlight
This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:
- Download and install Highlight from highlightai.com/download
- Navigate to the plugins tab and select "Add Custom Plugin"
-
Configure the plugin with the settings below
Plugin Name
Search Engine with RAG and MCPCommand (node, npx, python, etc.)Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
- Enable "Start Automatically" if you want the plugin to start when Highlight launches
From the repository
- Python 3.13+
- Poetry (optional, for development)
- API keys for Exa and FireCrawl
- (Optional) Ollama installed locally
- (Optional) OpenAI API key
Claude Desktop / Cursor
Paste into your MCP client config file to install this server.
{
"mcpServers": {
"search engine with rag and mcp": {
"search-engine-with-rag-and-mcp": {
"command": "python",
"args": [
"-m",
"src.core.main",
"your search query"
]
}
}
}
}McpServers
{
"search-engine-with-rag-and-mcp": {
"command": "python",
"args": [
"-m",
"src.core.main",
"your search query"
]
}
}A powerful search engine that combines LangChain, Model Context Protocol (MCP), Retrieval-Augmented Generation (RAG), and Ollama to create an agentic AI system capable of searching the web, retrieving information, and providing relevant answers.
Features
- Web search capabilities using the Exa API
- Web content retrieval using FireCrawl
- RAG (Retrieval-Augmented Generation) for more relevant information extraction
- MCP (Model Context Protocol) server for standardized tool invocation
- Support for both local LLMs via Ollama and cloud-based LLMs via OpenAI
- Flexible architecture supporting direct search, agent-based search, or server mode
- Comprehensive error handling and graceful fallbacks
- Python 3.13+ with type hints
- Asynchronous processing for efficient web operations
Architecture
This project integrates several key components:
1. Search Module: Uses Exa API to search the web and FireCrawl to retrieve content
2. RAG Module: Embeds documents, chunks them, and stores them in a FAISS vector store
3. MCP Server: Provides a standardized protocol for tool invocation
4. Agent: LangChain-based agent that uses the search and RAG capabilities
Project Structure
search-engine-with-rag-and-mcp/
├── LICENSE # MIT License
├── README.md # Project documentation
├── data/ # Data directories
├── docs/ # Documentation
│ └── env_template.md # Environment variables documentation
├── logs/ # Log files directory (auto-created)
├── src/ # Main package (source code)
│ ├── __init__.py
│ ├── core/ # Core functionality
│ │ ├── __init__.py
│ │ ├── main.py # Main entry point
│ │ ├── search.py # Web search module
│ │ ├── rag.py # RAG implementation
│ │ ├── agent.py # LangChain agent
│ │ └── mcp_server.py # MCP server implementation
│ └── utils/ # Utility modules
│ ├── __init__.py
│ ├── env.py # Environment variable loading
│ └── logger.py # Logging configuration
├── pyproject.toml # Poetry configuration
├── requirements.txt # Project dependencies
└── tests/ # Test directory
Getting Started
Prerequisites
- Python 3.13+
- Poetry (optional, for development)
- API keys for Exa and FireCrawl
- (Optional) Ollama installed locally
- (Optional) OpenAI API key
Installation
1. Clone the repository
git clone https://github.com/yourusername/search-engine-with-rag-and-mcp.git
cd search-engine-with-rag-and-mcp
2. Install dependencies
```bash
Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
