BerriAI/litellm · error · HTTPException

internal_server_error

internal_server_error

Error message

An unexpected error occurred: {e}

What it means

This is the catch-all handler at the bottom of POST /mcp-rest/tools/call: any exception the typed handlers (permissions, guardrails, BYOK, upstream auth) did not claim becomes HTTP 500 with error code internal_server_error. The proxy logs the full traceback via verbose_logger.exception("Unexpected error in MCP tool call"); the f-string message embeds the original exception text and is the starting point for diagnosis.

Solutions

  1. Read the proxy logs for the "Unexpected error in MCP tool call" traceback — the response message alone is rarely enough.
  2. Reproduce once with debug logging (LITELLM_LOG=DEBUG) to capture the full stack.
  3. Sanitize arguments to JSON primitives (strings, numbers, booleans, lists, dicts) before sending.
  4. If the traceback points inside litellm itself, open a GitHub issue with the stack trace and the tool-call shape.

Example fix

# before
await client.post("/mcp-rest/tools/call", json={
    "server_id": sid, "name": "create_event",
    "arguments": {"when": datetime.now()},  # datetime is not JSON-serializable
})

# after
await client.post("/mcp-rest/tools/call", json={
    "server_id": sid, "name": "create_event",
    "arguments": {"when": datetime.now().isoformat()},
})
Defensive patterns

Strategy: try-catch

Validate before calling

import json

def ensure_serializable(arguments: dict) -> dict:
    try:
        json.dumps(arguments)
    except TypeError as e:
        raise ValueError(f"arguments are not JSON-serializable: {e}") from e
    return arguments

ensure_serializable(payload["arguments"])

Try / catch

try:
    resp = await client.post(f"{base}/mcp-rest/tools/call", json=payload)
    resp.raise_for_status()
except httpx.HTTPStatusError as e:
    if e.response.status_code == 500:
        d = e.response.json().get("detail", {})
        # the message embeds the original exception; correlate with proxy logs,
        # retry at most once for transient-looking causes (network/DB)
        capture_for_postmortem(d.get("message"))
        return await call_with_backoff(payload, retries=1)
    raise

Prevention

When it happens

Trigger: Bugs in tool dispatch or response serialization; arguments containing objects that cannot be JSON-serialized; database or network failures mid-call; exceptions raised by custom callbacks or the upstream MCP SDK that no earlier handler catches.

Common situations: Right after upgrading litellm; passing datetime/set/custom objects inside arguments; flaky upstream connections during a call.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/750814912210030a. Report an issue: GitHub.

Appendix: source

Thrown at litellm/proxy/_experimental/mcp_server/rest_endpoints.py:1089

                    "message": str(e),
                    "guardrail_name": getattr(e, "guardrail_name", None),
                },
            )
        except MCPUpstreamAuthError as e:
            # A client-forwarded pass-through upstream 401 from either the direct or the virtual call
            # branch. Relay it as a 401 + WWW-Authenticate so the MCP client can re-run upstream OAuth,
            # and log at info: an expected caller-must-reauth signal, not an operator-actionable error.
            verbose_logger.info("MCP tool call relaying upstream HTTP %s", e.status_code)
            raise _relay_upstream_auth_http_exception(e, request)
        except HTTPException as e:
            # Locally generated denials (tool/server permission, IP filtering, BYOK) stay at error level
            # so restriction probing keeps full monitoring visibility; the relayed upstream 401 above is
            # the only status demoted to info.
            verbose_logger.error("HTTPException in MCP tool call: %s", e)
            raise e
        except Exception as e:
            verbose_logger.exception("Unexpected error in MCP tool call: %s", e)
            raise HTTPException(
                status_code=500,
                detail={
                    "error": "internal_server_error",
                    "message": f"An unexpected error occurred: {e}",
                },
            )

    ########################################################
    # MCP Connection testing routes
    # /health -> Test if we can connect to the MCP server
    # /health/tools/list -> List tools from MCP server
    # For these routes users will dynamically pass the MCP connection params, they don't need to be on the MCP registry
    ########################################################
    from litellm.proxy.management_endpoints.mcp_management_endpoints import (
        NewMCPServerRequest,
        _inherit_credentials_from_existing_server,
    )

View on GitHub (pinned to 77b7c6c40c)