{"record":{"id":"3602a6227de3be07","repo":"BerriAI/litellm","slug":"failed-to-parse-response-e","errorCode":null,"errorMessage":"Failed to parse response: {e}","messagePattern":"Failed to parse response: (.+?)","errorType":"http","errorClass":"SagemakerError","httpStatus":null,"severity":"error","filePath":"litellm/llms/sagemaker/embedding/transformation.py","lineNumber":99,"sourceCode":"\n    def transform_embedding_response(\n        self,\n        model: str,\n        raw_response: Response,\n        model_response: \"EmbeddingResponse\",\n        logging_obj: Any,\n        api_key: str | None = None,\n        request_data: dict = {},\n        optional_params: dict = {},\n        litellm_params: dict = {},\n    ) -> \"EmbeddingResponse\":\n        \"\"\"\n        Transform embedding response for Hugging Face models on SageMaker\n        \"\"\"\n        try:\n            response_data: Final = raw_response.json()\n        except Exception as e:\n            raise SagemakerError(\n                message=f\"Failed to parse response: {e}\",\n                status_code=raw_response.status_code,\n            )\n\n        # Handle both raw array format (TEI) and wrapped format (standard HF)\n        if isinstance(response_data, list):\n            # TEI and some HF models return raw embedding arrays directly\n            embeddings = response_data\n        elif isinstance(response_data, dict) and \"embedding\" in response_data:\n            # Standard HF format with \"embedding\" key\n            embeddings = response_data[\"embedding\"]\n        else:\n            raise SagemakerError(\n                status_code=500,\n                message=f\"Unexpected response format. Expected list or dict with 'embedding' key, got: {type(response_data).__name__}\",\n            )\n\n        if not isinstance(embeddings, list):","sourceCodeStart":81,"sourceCodeEnd":117,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/llms/sagemaker/embedding/transformation.py#L81-L117","documentation":"For Hugging Face/TEI embeddings on SageMaker, LiteLLM calls raw_response.json() to parse the endpoint body. If the body is not valid JSON (HTML error page, plain-text error, empty body), it raises SagemakerError carrying the HTTP status code of the original response and the parse exception text.","triggerScenarios":"The SageMaker embedding endpoint returns 4xx/5xx with an HTML or plain-text body, the container crashes mid-response, or a proxy/gateway between the client and SageMaker returns a non-JSON error page.","commonSituations":"TEI container OOM/timeout returning an HTML 502; wrong ContentType causing the container to reject with a text error; endpoints fronted by a corporate proxy that strips or replaces responses.","solutions":["Check raw_response.status_code in the raised error: 4xx points at the request payload, 5xx at the container.","Invoke the endpoint directly with boto3 and inspect the raw Body bytes to see what is actually returned.","For 5xx, check SageMaker endpoint CloudWatch logs for container crashes (OOM, model load failures).","If a proxy is in play, bypass it or configure it to pass JSON responses through unmodified."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"from litellm import SagemakerError\n\ntry:\n    resp = litellm.embedding(model='sagemaker/tei', input=texts)\nexcept SagemakerError as e:\n    if 'Failed to parse response' in str(e):\n        # body was not JSON - inspect endpoint container logs / CloudWatch\n        log.error('non-JSON body from embedding endpoint, status=%s', e.status_code)\n    raise","preventionTips":["Monitor the endpoint container for OOMs when raising batch sizes.","Keep an httpx/boto3 canary that asserts the endpoint returns parseable JSON."],"tags":["sagemaker","embedding","response-parsing","json"],"backgroundTag":"invalid-json-response","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}