{"record":{"id":"4dc563b0584dc16f","repo":"BerriAI/litellm","slug":"sagemakerexception-the-value-of-n-must-be-0","errorCode":null,"errorMessage":"SagemakerException - the value of 'n' must be > 0 and <= 2 for sagemaker endpoints","messagePattern":"SagemakerException - the value of 'n' must be > 0 and <= 2 for sagemaker endpoints","errorType":"exception","errorClass":"BadRequestError","httpStatus":400,"severity":"error","filePath":"litellm/litellm_core_utils/exception_mapping_utils.py","lineNumber":985,"sourceCode":"def _map_sagemaker_exception(\n    *,\n    model: str,\n    original_exception: _ProviderHTTPException,\n    custom_llm_provider: str,\n    error_str: str,\n    exception_type: str,\n    exception_provider: str,\n    extra_information: str,\n) -> None:\n    if \"Unable to locate credentials\" in error_str:\n        raise BadRequestError(\n            message=f\"litellm.BadRequestError: SagemakerException - {error_str}\",\n            model=model,\n            llm_provider=\"sagemaker\",\n            response=getattr(original_exception, \"response\", None),\n        )\n    elif \"Input validation error: `best_of` must be > 0 and <= 2\" in error_str:\n        raise BadRequestError(\n            message=\"SagemakerException - the value of 'n' must be > 0 and <= 2 for sagemaker endpoints\",\n            model=model,\n            llm_provider=\"sagemaker\",\n            response=getattr(original_exception, \"response\", None),\n        )\n    elif (\n        \"`inputs` tokens + `max_new_tokens` must be <=\" in error_str\n        or \"instance type with more CPU capacity or memory\" in error_str\n    ):\n        raise ContextWindowExceededError(\n            message=f\"SagemakerException - {error_str}\",\n            model=model,\n            llm_provider=\"sagemaker\",\n            response=getattr(original_exception, \"response\", None),\n        )\n    elif hasattr(original_exception, \"status_code\"):\n        if original_exception.status_code == 500:\n            raise ServiceUnavailableError(","sourceCodeStart":967,"sourceCodeEnd":1003,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/litellm_core_utils/exception_mapping_utils.py#L967-L1003","documentation":"When the SageMaker error string contains 'Input validation error: `best_of` must be > 0 and <= 2', litellm raises litellm.BadRequestError with the friendly message 'SagemakerException - the value of 'n' must be > 0 and <= 2 for sagemaker endpoints'. SageMaker HuggingFace endpoints cap the number of returned completions at 2; litellm translates a request for more into this deterministic client error instead of the raw provider text.","triggerScenarios":"Calling litellm.completion on a sagemaker/ model with n (or best_of) greater than 2; OpenAI-compatible defaults or config copied from OpenAI usage where n is commonly higher; code that scales n with fan-out logic ignoring provider caps.","commonSituations":"Porting OpenAI multi-choice generation (n=5) code to SageMaker-hosted models; prompt-evaluation harnesses that sample k completions; shared request-building code across providers that hardcodes n.","solutions":["Set n <= 2 (and best_of <= 2) for sagemaker/ calls","If you need more samples, loop the call k times instead of one call with n=k","Gate n per provider in shared code: cap it when provider is sagemaker","Consider bedrock/ or openai/ providers if many samples per call is a hard requirement"],"exampleFix":"# before\nresp = litellm.completion(model=\"sagemaker/my-endpoint\", messages=msgs, n=5)\n\n# after\nn = min(n, 2) if model.startswith(\"sagemaker/\") else n\nresp = litellm.completion(model=\"sagemaker/my-endpoint\", messages=msgs, n=n)","handlingStrategy":"validation","validationCode":"def safe_n(n: int, model: str) -> int:\n    if model.startswith(\"sagemaker/\"):\n        if n > 2:\n            raise ValueError(\"sagemaker endpoints allow n <= 2\")\n    return n","typeGuard":"import litellm\n\ndef is_sagemaker_n_error(e: Exception) -> bool:\n    return isinstance(e, litellm.BadRequestError) and \"must be > 0 and <= 2\" in str(e)","tryCatchPattern":null,"preventionTips":["Cap n and best_of at 2 for sagemaker/ models at the request-building layer","Use multiple calls when you need more than 2 samples","Encode provider capability limits (like n caps) in provider config, not scattered call sites"],"tags":["sagemaker","aws","bad-request","sampling","validation"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}