Ragscore MCP Server

Generate QA datasets & evaluate RAG systems in 2 commands 🔒 Privacy-First • ⚡ Lightning Fast • 🤖 Any LLM • 🏠 Local or Cloud • 🌍 Multilingual

People connecting AI & ML tools to Claude, Cursor, VS Code, or another MCP client. The project is written in Python.

VERIFIED ACTIVE

LAST COMMIT 2026-05-29 · ★ 15 · #50 OF 95 MAINTAINED AI & ML · VERIFIED 2026-08-25

Apache-2.0 · Python servers · how we verify → /methodology

01 · Install Ragscore

before you install - you'll need

Set OPENAI_API_KEY, RAGSCORE_CHUNK_SIZE, RAGSCORE_QUESTIONS_PER_CHUNK, RAGSCORE_WORK_DIR, RAGSCORE_NO_TELEMETRY before connecting.

Claude Code

claude mcp add hzyai-ragscore -- uvx ragscore

Claude Desktop / Cursor / VS Code - add to config

{
  "mcpServers": {
    "hzyai-ragscore": {
      "command": "uvx",
      "args": [
        "ragscore"
      ]
    }
  }
}

Same JSON for Cursor. For VS Code, rename the top-level key from `mcpServers` to `servers`.

Using another client? Same JSON, different key

Claude Desktop · mcpServers

Cursor · mcpServers

VS Code · servers

Windsurf · mcpServers

Zed · context_servers

Cline · mcpServers

Roo Code · mcpServers

Continue · mcpServers

LibreChat · mcpServers

Gemini CLI · mcpServers

Codex CLI · mcp_servers

Full setup guides: every client.

02 · Evidence

Security posture

What to check before giving this server access to your agent - from the registry, GitHub, and our own probes. We don't score safety; we show what's verifiable.

runs as local process (stdio) - runs on your machine with your user's permissions

license Apache-2.0 - declared in the repository

pypi package ragscore - check the name against the project README before installing (PyPI has no namespace ownership)

registry namespace io.github.HZYAI is GitHub-verified and matches the repo owner

03 · What Ragscore can do

Prose above is summarized from the project's README and registry record - no invented capabilities.

Latest releases

v0.7.0 · 2026-02-15

What's New · Detailed Multi-Metric Evaluation · Add detailed=True to get 5 diagnostic dimensions per answer - same single LLM call: · Correctness - semantic match to golden answer · Completeness - covers all key points…

04 · Who maintains Ragscore

RAGScore is maintained by hzyai. It's the only MCP server we track from this author; the repo dates to Dec 2025.

05 · Facts

repository: github.com/hzyai/ragscore
category: AI & ML - ranked #50 of 95 actively-maintained AI & ML servers as of 2026-08-25.
registry: io.github.HZYAI/ragscore (active, first published 2026-02-11 · 3 versions)
packages: pypi:ragscore

06 · Ragscore FAQ

What is Ragscore?

Generate QA datasets & evaluate RAG systems in 2 commands 🔒 Privacy-First • ⚡ Lightning Fast • 🤖 Any LLM • 🏠 Local or Cloud • 🌍 Multilingual

Is Ragscore still maintained?

Yes - as of 2026-08-25, its last commit was 2026-05-29. We re-verify nightly.

How do I install Ragscore?

Run `uvx ragscore`. The README documents 5 environment variables (OPENAI_API_KEY, RAGSCORE_CHUNK_SIZE, RAGSCORE_QUESTIONS_PER_CHUNK…) to set first. Set OPENAI_API_KEY, RAGSCORE_CHUNK_SIZE, RAGSCORE_QUESTIONS_PER_CHUNK, RAGSCORE_WORK_DIR, RAGSCORE_NO_TELEMETRY before connecting. You can also paste the ready-made client config above.

Does Ragscore run locally?

Yes - it's a stdio server: it runs on your machine (via uvx) with your user's permissions. Your data stays local unless the server itself calls external APIs.

07 · Alternatives to Ragscore

Alternatives to Ragscore

Maintained AI & ML servers if Ragscore isn't the fit.

Pairs well with

Servers that cover what Ragscore doesn't - only shown when the pairing reason fits the companion.

More AI & ML MCP servers · Mcpbridge · DEVONthink MCP (istefox) · Meta Prompt MCP · RAGMap MCP · Lynx

More Python MCP servers · Grad Agent · Tidal MCP · Disney Lorcana TCG Collection · T-Bank MCP · IdentArk Gateway · see all