BerriAI/litellm · error · SagemakerError

Failed to parse response: {e}

Error message

Failed to parse response: {e}

What it means

For Hugging Face/TEI embeddings on SageMaker, LiteLLM calls raw_response.json() to parse the endpoint body. If the body is not valid JSON (HTML error page, plain-text error, empty body), it raises SagemakerError carrying the HTTP status code of the original response and the parse exception text.

Source

Thrown at litellm/llms/sagemaker/embedding/transformation.py:99

    def transform_embedding_response(
        self,
        model: str,
        raw_response: Response,
        model_response: "EmbeddingResponse",
        logging_obj: Any,
        api_key: str | None = None,
        request_data: dict = {},
        optional_params: dict = {},
        litellm_params: dict = {},
    ) -> "EmbeddingResponse":
        """
        Transform embedding response for Hugging Face models on SageMaker
        """
        try:
            response_data: Final = raw_response.json()
        except Exception as e:
            raise SagemakerError(
                message=f"Failed to parse response: {e}",
                status_code=raw_response.status_code,
            )

        # Handle both raw array format (TEI) and wrapped format (standard HF)
        if isinstance(response_data, list):
            # TEI and some HF models return raw embedding arrays directly
            embeddings = response_data
        elif isinstance(response_data, dict) and "embedding" in response_data:
            # Standard HF format with "embedding" key
            embeddings = response_data["embedding"]
        else:
            raise SagemakerError(
                status_code=500,
                message=f"Unexpected response format. Expected list or dict with 'embedding' key, got: {type(response_data).__name__}",
            )

        if not isinstance(embeddings, list):

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Check raw_response.status_code in the raised error: 4xx points at the request payload, 5xx at the container.
2. Invoke the endpoint directly with boto3 and inspect the raw Body bytes to see what is actually returned.
3. For 5xx, check SageMaker endpoint CloudWatch logs for container crashes (OOM, model load failures).
4. If a proxy is in play, bypass it or configure it to pass JSON responses through unmodified.

from litellm import SagemakerError

try:
    resp = litellm.embedding(model='sagemaker/tei', input=texts)
except SagemakerError as e:
    if 'Failed to parse response' in str(e):
        # body was not JSON - inspect endpoint container logs / CloudWatch
        log.error('non-JSON body from embedding endpoint, status=%s', e.status_code)
    raise

When it happens

Trigger: The SageMaker embedding endpoint returns 4xx/5xx with an HTML or plain-text body, the container crashes mid-response, or a proxy/gateway between the client and SageMaker returns a non-JSON error page.

Common situations: TEI container OOM/timeout returning an HTML 502; wrong ContentType causing the container to reject with a text error; endpoints fronted by a corporate proxy that strips or replaces responses.

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Unexpected response format. Expected list or dict with 'embe
• HF response not in expected format - {embeddings}
• {raw_response.text}
• Mavvrik FOCUS destination: response missing 'url' field: {re
• AgentCore: Failed to read/parse JSON response body: {e}

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/3602a6227de3be07. Report an issue: GitHub.