monarch-mcp-custom/PROBLEM.md

PROBLEM: Monarch MCP Server SSE Routing and Handshake Failures

Issue Description

The Monarch MCP server, implemented using FastMCP and Starlette, is experiencing persistent 404 Not Found, TypeError, or 400 Bad Request errors when AI agent clients attempt to establish an SSE connection at http://docker.local.ben.io:8070/mcp. While the /health endpoint is functioning correctly, the MCP handshake fails to complete or routes to non-existent endpoints.

Root Cause Analysis (Suspected)

The primary issue stems from the complexity of mounting a FastMCP HTTP application within a Starlette wrapper.

1. Endpoint Overlap: FastMCP.http_app() generates its own internal routing (e.g., /mcp or /sse). When this app is mounted at /mcp in a parent Starlette app, the resulting path becomes /mcp/mcp, leading to 404 errors on the expected /mcp path.
2. ASGI Signature Mismatches: Manual attempts to bridge the SseServerTransport with Starlette routes led to TypeError: 'NoneType' object is not callable or missing positional arguments (receive, send), indicating that the custom endpoint wrappers were not correctly following the ASGI/Starlette interface.
3. Transport Defaults: FastMCP default transport ("streamable-http") expects specific headers and session IDs that differ from standard MCP SSE implementations, causing 400 Bad Request: Missing session ID when using generic tools like curl.

Troubleshooting Steps Taken

1. Refactored Auth: Removed keyring from the Docker runtime to prevent headless environment crashes; moved to MONARCH_TOKEN environment variable.
2. Infrastructure Validation: Verified port 8070 is unique on the local.ben.io server.
3. Internal Logic Debugging:
   • Attempted mounting FastMCP at root (/) and sub-paths (/mcp).
   • Inspected FastMCP source code to identify internal attributes (_mcp_server, _lifespan).
   • Switched from manual SseServerTransport handling to mcp.http_app().
4. Endpoint Verification:
   • Confirmed /health returns 200 OK.
   • Tested /mcp with Accept: text/event-stream, which yielded 400 Bad Request or 406 Not Acceptable depending on the mount point.
5. Session Exploration: Investigated OpenCode session storage (~/.local/share/opencode/storage/session/) to understand how clients identify and connect to projects.

Current State

The server is running in a Komodo stack, healthy on port 8070, and visible at https://monarch-mcp.ext.ben.io. However, the handshake remains broken.

Resolution (Applied)

The fix was to match the working pattern from komodo-mcp-custom:

def create_app():
    mcp_app = mcp.http_app()
    routes = [
        Route("/health", health, methods=["GET"]),
        Mount("/", app=mcp_app),  # MCP handles /mcp endpoint internally
    ]
    return Starlette(routes=routes, lifespan=mcp_app.lifespan)

app = create_app()

Key changes:

1. Mount at root: Mount("/", app=mcp_app) lets FastMCP handle /mcp internally
2. Pass lifespan: lifespan=mcp_app.lifespan ensures proper task group initialization
3. Separate health route: /health is a standalone route in the parent Starlette app