{"record":{"id":"7f36ae512f99b100","repo":"BerriAI/litellm","slug":"sagemakerexception-original-exception-message","errorCode":null,"errorMessage":"SagemakerException - {original_exception.message}","messagePattern":"SagemakerException - (.+?)","errorType":"exception","errorClass":"ServiceUnavailableError","httpStatus":500,"severity":"error","filePath":"litellm/litellm_core_utils/exception_mapping_utils.py","lineNumber":1003,"sourceCode":"        raise BadRequestError(\n            message=\"SagemakerException - the value of 'n' must be > 0 and <= 2 for sagemaker endpoints\",\n            model=model,\n            llm_provider=\"sagemaker\",\n            response=getattr(original_exception, \"response\", None),\n        )\n    elif (\n        \"`inputs` tokens + `max_new_tokens` must be <=\" in error_str\n        or \"instance type with more CPU capacity or memory\" in error_str\n    ):\n        raise ContextWindowExceededError(\n            message=f\"SagemakerException - {error_str}\",\n            model=model,\n            llm_provider=\"sagemaker\",\n            response=getattr(original_exception, \"response\", None),\n        )\n    elif hasattr(original_exception, \"status_code\"):\n        if original_exception.status_code == 500:\n            raise ServiceUnavailableError(\n                message=f\"SagemakerException - {original_exception.message}\",\n                llm_provider=custom_llm_provider,\n                model=model,\n                response=httpx.Response(\n                    status_code=500,\n                    request=httpx.Request(method=\"POST\", url=\"https://api.openai.com/v1/\"),\n                ),\n            )\n        elif original_exception.status_code == 401:\n            raise AuthenticationError(\n                message=f\"SagemakerException - {original_exception.message}\",\n                llm_provider=custom_llm_provider,\n                model=model,\n                response=getattr(original_exception, \"response\", None),\n            )\n        elif original_exception.status_code == 400:\n            raise BadRequestError(\n                message=f\"SagemakerException - {original_exception.message}\",","sourceCodeStart":985,"sourceCodeEnd":1021,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/litellm_core_utils/exception_mapping_utils.py#L985-L1021","documentation":"litellm maps a SageMaker failure carrying status_code 500 to litellm.ServiceUnavailableError with 'SagemakerException - <original message>'. The SageMaker runtime (or the model server behind the endpoint) returned an internal server error: the container crashed on the payload, the model process OOMed, or invoke_endpoint hit an unexpected server-side fault.","triggerScenarios":"Calling a 'sagemaker/' model where the endpoint's container returns 500: payload deserialization crashing the custom inference code, OOM kills of the model process, broken model artifacts after a bad deploy, or transient runtime faults. The mapped error attaches a synthetic httpx.Response(500) as a placeholder, not the real SageMaker response.","commonSituations":"Custom inference scripts raising unhandled exceptions; under-provisioned instances under concurrent load; endpoints mid-update while traffic flows; malformed input that bypasses validation and crashes handler code.","solutions":["Retry with backoff — a single 500 is often transient (num_retries on the call or Router)","Check endpoint CloudWatch logs (/aws/sagemaker/Endpoints/<endpoint>) for the container stack trace","Fix or harden the inference handler if the trace shows an unhandled exception on your payload shape","Scale the endpoint (instance size/count) if OOM or saturation recurs"],"exampleFix":"# before\nresp = litellm.completion(model=\"sagemaker/my-endpoint\", messages=msgs)\n\n# after\nresp = litellm.completion(model=\"sagemaker/my-endpoint\", messages=msgs, num_retries=3)","handlingStrategy":"retry","validationCode":null,"typeGuard":"import litellm\n\ndef is_endpoint_500(e: Exception) -> bool:\n    return isinstance(e, litellm.ServiceUnavailableError)","tryCatchPattern":"import litellm\ntry:\n    resp = litellm.completion(model=\"sagemaker/ep\", messages=msgs)\nexcept litellm.ServiceUnavailableError:\n    # container-side failure: retry a bounded number of times, then alert\n    resp = litellm.completion(model=\"sagemaker/ep\", messages=msgs, num_retries=3)","preventionTips":["Bound retries; repeated 500s point to a crashing handler, not transient faults","Ship structured error responses from your inference code so 500s become 400s with messages","Monitor endpoint CloudWatch logs for container exceptions alongside client retry metrics"],"tags":["sagemaker","aws","http-500","service-unavailable","retryable"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}