BerriAI/litellm · error · BadRequestError

SagemakerException - the value of 'n' must be > 0 and <= 2 f

Error message

SagemakerException - the value of 'n' must be > 0 and <= 2 for sagemaker endpoints

What it means

When the SageMaker error string contains 'Input validation error: `best_of` must be > 0 and <= 2', litellm raises litellm.BadRequestError with the friendly message 'SagemakerException - the value of 'n' must be > 0 and <= 2 for sagemaker endpoints'. SageMaker HuggingFace endpoints cap the number of returned completions at 2; litellm translates a request for more into this deterministic client error instead of the raw provider text.

Source

Thrown at litellm/litellm_core_utils/exception_mapping_utils.py:985

def _map_sagemaker_exception(
    *,
    model: str,
    original_exception: _ProviderHTTPException,
    custom_llm_provider: str,
    error_str: str,
    exception_type: str,
    exception_provider: str,
    extra_information: str,
) -> None:
    if "Unable to locate credentials" in error_str:
        raise BadRequestError(
            message=f"litellm.BadRequestError: SagemakerException - {error_str}",
            model=model,
            llm_provider="sagemaker",
            response=getattr(original_exception, "response", None),
        )
    elif "Input validation error: `best_of` must be > 0 and <= 2" in error_str:
        raise BadRequestError(
            message="SagemakerException - the value of 'n' must be > 0 and <= 2 for sagemaker endpoints",
            model=model,
            llm_provider="sagemaker",
            response=getattr(original_exception, "response", None),
        )
    elif (
        "`inputs` tokens + `max_new_tokens` must be <=" in error_str
        or "instance type with more CPU capacity or memory" in error_str
    ):
        raise ContextWindowExceededError(
            message=f"SagemakerException - {error_str}",
            model=model,
            llm_provider="sagemaker",
            response=getattr(original_exception, "response", None),
        )
    elif hasattr(original_exception, "status_code"):
        if original_exception.status_code == 500:
            raise ServiceUnavailableError(

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Set n <= 2 (and best_of <= 2) for sagemaker/ calls
2. If you need more samples, loop the call k times instead of one call with n=k
3. Gate n per provider in shared code: cap it when provider is sagemaker
4. Consider bedrock/ or openai/ providers if many samples per call is a hard requirement

Example fix

# before
resp = litellm.completion(model="sagemaker/my-endpoint", messages=msgs, n=5)

# after
n = min(n, 2) if model.startswith("sagemaker/") else n
resp = litellm.completion(model="sagemaker/my-endpoint", messages=msgs, n=n)

def safe_n(n: int, model: str) -> int:
    if model.startswith("sagemaker/"):
        if n > 2:
            raise ValueError("sagemaker endpoints allow n <= 2")
    return n

import litellm

def is_sagemaker_n_error(e: Exception) -> bool:
    return isinstance(e, litellm.BadRequestError) and "must be > 0 and <= 2" in str(e)

When it happens

Trigger: Calling litellm.completion on a sagemaker/ model with n (or best_of) greater than 2; OpenAI-compatible defaults or config copied from OpenAI usage where n is commonly higher; code that scales n with fan-out logic ignoring provider caps.

Common situations: Porting OpenAI multi-choice generation (n=5) code to SageMaker-hosted models; prompt-evaluation harnesses that sample k completions; shared request-building code across providers that hardcodes n.

Related errors

• litellm.BadRequestError: SagemakerException - {error_str}
• SagemakerException - {error_str}
• SagemakerException - {original_exception.message}
• model parameter is required but was None. Please provide a v
• Invalid guardrailConfig={raw_guardrail_config}. Expected for

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/4dc563b0584dc16f. Report an issue: GitHub.