Skip to content

feat(valkey-mcp-server): V2 — GLIDE migration, 12 focused tools#7

Closed
Jonathan-Improving wants to merge 21 commits into
mainfrom
feature/mcp/v2
Closed

feat(valkey-mcp-server): V2 — GLIDE migration, 12 focused tools#7
Jonathan-Improving wants to merge 21 commits into
mainfrom
feature/mcp/v2

Conversation

@Jonathan-Improving
Copy link
Copy Markdown

@Jonathan-Improving Jonathan-Improving commented Apr 24, 2026

Valkey MCP Server V2 — Focused Search + JSON + Command Runner Tools

Major rewrite of the Valkey MCP server: migrates from valkey-py to Valkey-GLIDE, replaces ~40+ generic data-type tools with 12 purpose-built tools (4 AI Search + 5 JSON Intelligence + 3 Command Runners), and adds a multi-provider embeddings layer.

What Changed

Architecture:

  • Migrated from valkey-py to valkey-glide (async-native GLIDE client)
  • Replaced ~40 per-data-type tool wrappers with 12 focused tools
  • Added multi-provider embeddings (Bedrock, OpenAI, Ollama, Hash) with automatic fallback
  • Native stdout fd redirect to prevent GLIDE's Rust logger from corrupting MCP stdio transport
  • Configurable GLIDE client logging via VALKEY_GLIDE_LOG_LEVEL env var

New Tools:

# Tool Category
1 manage_index AI Search — FT.CREATE / DROP / INFO / _LIST with structured JSON input
2 add_documents AI Search — Embed → binary pack → HSET pipeline, auto-creates index
3 search AI Search — Unified semantic, text, hybrid, find-similar with mode auto-detection
4 aggregate AI Search — FT.AGGREGATE structured pipeline builder
5 json_get JSON Intelligence — Typed JSON.GET with path
6 json_set JSON Intelligence — Typed JSON.SET with path + optional TTL
7 json_arrappend JSON Intelligence — Append to JSON array
8 json_arrpop JSON Intelligence — Pop from JSON array
9 json_arrtrim JSON Intelligence — Trim JSON array
10 valkey_read Command Runner — Read-only commands (GET, HGETALL, LRANGE, etc.)
11 valkey_write Command Runner — Mutating commands (SET, HSET, LPUSH, etc.)
12 valkey_admin Command Runner — Destructive commands (FLUSHDB, DEBUG, etc.) — opt-in via VALKEY_ADMIN_ENABLED

3-Tier Command Safety:

  • valkey_read — always available, read-only commands
  • valkey_write — available unless readonly mode is active
  • valkey_admin — disabled by default, requires VALKEY_ADMIN_ENABLED=true

Removed: All generic data-type tools (Strings, Lists, Sets, Sorted Sets, Hashes, Streams, Bitmaps, HyperLogLog, misc, server_management) — these performed worse than raw CLI in benchmarks.

Safety & Reliability

  • @tool_errors decorator catches RequestError only — programming errors propagate for debugging
  • @readonly_guard decorator enforces readonly mode on mutating tools
  • asyncio.Lock on GLIDE client singleton (double-check pattern)
  • threading.Lock on embeddings provider singleton
  • Defensive atexit shutdown handler for GLIDE client + Ollama httpx client
  • Deferred STS credential validation (runs in executor on first Bedrock embedding call)
  • File-based logging with loguru for post-mortem analysis

Testing

  • 137 unit tests (mocked)
  • 55 live integration tests against real Valkey + Ollama
  • TLS integration tests with CA cert verification and password auth
  • Error case tests covering all previously-discovered crash scenarios
  • Tests organized: tests/ (unit) and tests/integration/ (live)
  • pytest-timeout to prevent hanging tests

Documentation

  • README fully rewritten with quickstart, configuration tables, troubleshooting
  • Tool docstrings with Args/Returns/Examples
  • JSON.SET TTL non-atomicity documented

Related

  • Jira: AEA-355
  • Plan: docs/AEA-355-plan.md

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.

Copy link
Copy Markdown

@jbrinkman jbrinkman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: Valkey MCP Server V2 — GLIDE Migration + 9 Focused Tools

Summary

This is a well-structured major rewrite that replaces ~40 generic data-type tools with 9 purpose-built tools for AI Search and JSON operations, migrating from valkey-py to valkey-glide. The architecture is significantly cleaner, the tool surface is more focused, and the pre-validation pattern to prevent GLIDE native crashes is a smart defensive measure.


🟢 What's Good

  • Pre-validation pattern — Checking index_exists() and _require_array() before GLIDE calls prevents native crashes. This is well-tested in test_error_cases_live.py.
  • Structured JSON input — The aggregate pipeline builder eliminates DSL syntax errors from AI agents. The LOAD field auto-detection is particularly well done.
  • Multi-provider embeddings — Clean factory pattern with Bedrock/OpenAI/Ollama/Hash providers. Good separation of concerns.
  • Test coverage — 110 unit tests + 35 live integration tests. Error case tests specifically targeting previously-discovered crash scenarios is excellent.
  • README rewrite — Troubleshooting table, quickstart, and configuration tables are much more useful than before.
  • decode_value utility — Recursive bytes decoding handles GLIDE's raw responses cleanly.

🔴 Issues to Address

  1. connection.py — Global singleton race condition (see inline comment)

    • get_client() has no locking. Concurrent coroutines can create duplicate GLIDE clients.
  2. main.pyatexit handler will crash (see inline comment)

    • asyncio.run(close_client()) fails when an event loop is already running at shutdown.

🟡 Issues to Consider

  1. providers.pyOllamaEmbeddings httpx client never closed — Resource leak over time.

  2. providers.pyBedrockEmbeddings.__init__ makes synchronous STS call — Blocks event loop at startup.

  3. search_aggregate.py — Wildcard * query rejected — This is a valid FT.AGGREGATE pattern. The restriction limits usability.

  4. search_query.py_find_similar pagination buglimit + 1 propagated to offset causes off-by-one when paginating.

  5. json.pyJSON.SET + EXPIRE not atomic — TTL race condition if process crashes between the two calls.

  6. json.py — Bare Exception catching — Swallows programming errors. Consider catching RequestError specifically.

  7. providers.pyHashEmbeddings cycles 32 bytes for dimensions > 32 — Produces repeated patterns in test vectors.

  8. connection.py — Synchronous file read for CA cert — Minor, but blocks event loop if cert is on network storage.


📝 Minor Notes

  • The docs/AEA-355-plan.md is a great internal document but probably shouldn't ship in the published package. Consider adding it to .gitignore or a docs/internal/ directory.
  • pyproject.toml adds boto3, openai, and httpx as hard dependencies. Users who only need JSON tools or text search will pull in AWS SDK and OpenAI SDK unnecessarily. Consider making these optional extras: pip install awslabs.valkey-mcp-server[bedrock], [openai], [ollama].
  • The hybrid_weight parameter in search() is documented as "advisory — reserved for future weighted scoring" but it's used for mode auto-detection (!= 0.5 triggers hybrid). This is confusing — either implement weighted scoring or remove the auto-detection behavior.

Verdict

The architecture and tool design are solid. The two 🔴 issues (singleton race condition and atexit crash) should be fixed before merge. The 🟡 items are worth addressing but aren't blockers.

Comment thread src/valkey-mcp-server/awslabs/valkey_mcp_server/main.py
Comment thread src/valkey-mcp-server/awslabs/valkey_mcp_server/tools/json.py
Comment thread src/valkey-mcp-server/awslabs/valkey_mcp_server/tools/json.py
@Jonathan-Improving Jonathan-Improving force-pushed the feature/mcp/v2 branch 2 times, most recently from 9107f21 to 78127b8 Compare April 24, 2026 18:31
Comment thread src/valkey-mcp-server/awslabs/valkey_mcp_server/main.py Fixed
Comment thread src/valkey-mcp-server/awslabs/valkey_mcp_server/common/connection.py Outdated
Comment thread src/valkey-mcp-server/awslabs/valkey_mcp_server/common/connection.py Outdated
Comment thread src/valkey-mcp-server/awslabs/valkey_mcp_server/common/connection.py Outdated
@Jonathan-Improving Jonathan-Improving changed the title feat(valkey-mcp-server): V2 — GLIDE migration, 9 focused search + JSON tools feat(valkey-mcp-server): V2 — GLIDE migration, 12 focused tools Apr 24, 2026
Comment thread src/valkey-mcp-server/awslabs/valkey_mcp_server/common/__init__.py Outdated
Comment thread src/valkey-mcp-server/awslabs/valkey_mcp_server/embeddings/providers.py Outdated
Comment thread src/valkey-mcp-server/awslabs/valkey_mcp_server/embeddings/providers.py Outdated
Comment thread src/valkey-mcp-server/awslabs/valkey_mcp_server/tools/search_aggregate.py Outdated
Comment thread src/valkey-mcp-server/tests/test_command_runners.py
Comment thread src/valkey-mcp-server/tests/test_connection.py
Comment thread src/valkey-mcp-server/tests/test_connection.py Outdated
Comment thread src/valkey-mcp-server/tests/test_search_query.py
Comment thread src/valkey-mcp-server/README.md
Copy link
Copy Markdown

@edlng edlng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the note is added to the docstring for hybrid search, lgtm!

…ine default dimension logic, and refactor AWS credential validation.
@Jonathan-Improving
Copy link
Copy Markdown
Author

Closing, superceded by duplicated PR on AWS Labs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants