BerriAI/litellm · error · ValueError

No model could be resolved for MCP sampling. Please configur

Error message

No model could be resolved for MCP sampling. Please configure 'default_mcp_sampling_model' in your LiteLLM configuration.

What it means

When an upstream MCP server sends a sampling request (createMessage), LiteLLM must choose a chat model to fulfill it. Resolution order: the client's ModelPreferences hints, then priority weights, then the caller-provided default, then the first model in the proxy router/litellm.model_list, then litellm.default_mcp_sampling_model. If every step comes up empty — no models deployed and no default configured — this ValueError is raised.

Source

Thrown at litellm/proxy/_experimental/mcp_server/sampling_handler.py:162

            default_model,
        )
        return default_model
    # Fall back to first available model
    if available_model_names:
        verbose_logger.debug(
            "MCP sampling model resolution: no default configured, falling back to first available model '%s'",
            available_model_names[0],
        )
        return available_model_names[0]
    # Last resort - use LiteLLM default or raise error
    default_sampling_model: Final[str | None] = getattr(litellm, "default_mcp_sampling_model", None)
    if default_sampling_model:
        verbose_logger.debug(
            "MCP sampling model resolution: using litellm.default_mcp_sampling_model='%s'",
            default_sampling_model,
        )
        return default_sampling_model
    raise ValueError(
        "No model could be resolved for MCP sampling. Please configure 'default_mcp_sampling_model' in your LiteLLM configuration."
    )


def _has_priorities(model_preferences: "ModelPreferences") -> bool:
    """Return True if any priority weight is set (non-None and > 0)."""
    return any(
        (getattr(model_preferences, attr, None) or 0) > 0
        for attr in ("costPriority", "speedPriority", "intelligencePriority")
    )


class _ScoredModel(NamedTuple):
    name: str
    cost: float
    max_output: float
    output_tps: float

View on GitHub (pinned to 77b7c6c40c)

Solutions

Set a default sampling model: litellm_settings.default_mcp_sampling_model: <deployment name> in the proxy config (applied as litellm.default_mcp_sampling_model), or set that attribute in code.
Or add at least one model deployment to model_list so the first-available fallback works.
Or have the MCP client send ModelPreferences hints that match a deployed model name.

Example fix

# before (config.yaml)
litellm_settings: {}

# after
litellm_settings:
  default_mcp_sampling_model: openai/gpt-4o-mini

Defensive patterns

Strategy: validation

Validate before calling

def sampling_model_resolvable(default: str | None = None) -> bool:
    import litellm
    try:
        from litellm.proxy.proxy_server import llm_router
        if llm_router is not None and llm_router.get_model_names():
            return True
    except Exception:
        pass
    return bool(default or getattr(litellm, "default_mcp_sampling_model", None))

assert sampling_model_resolvable(), "deploy a model or set default_mcp_sampling_model before enabling sampling"

Try / catch

try:
    result = await handle_sampling_request(create_message_request)
except ValueError as e:
    if "default_mcp_sampling_model" in str(e):
        # configuration problem, not transient: surface to operator, do not retry
        raise ConfigError("set litellm_settings.default_mcp_sampling_model or add a model deployment") from e
    raise

Prevention

If the proxy serves MCP servers that may issue sampling requests, always set default_mcp_sampling_model.
Smoke-test sampling after deployment with a minimal createMessage request.
Verify model_list actually loaded (router.get_model_names() is non-empty) before going live.

When it happens

Trigger: An MCP server invokes sampling while the proxy runs with an empty model_list (for example a pure MCP gateway with zero LLM deployments) and default_mcp_sampling_model is unset; router initialization failed so get_model_names() returns nothing.

Common situations: Deploying litellm-proxy solely as an MCP gateway without LLM deployments; local testing without a config file; config typos that leave model_list empty.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/95183fd7a2c221c6. Report an issue: GitHub.