BerriAI/litellm · error · SagemakerError

Unexpected response format. Expected list or dict with 'embe

Error message

Unexpected response format. Expected list or dict with 'embedding' key, got: {type(response_data).__name__}

What it means

After successfully JSON-parsing a SageMaker embedding response, LiteLLM accepts either a raw list (TEI style) or a dict containing an 'embedding' key (HF inference-dict style). Anything else - a dict without 'embedding', a bare string, a number - raises SagemakerError 500 naming the offending Python type.

Source

Thrown at litellm/llms/sagemaker/embedding/transformation.py:112

        Transform embedding response for Hugging Face models on SageMaker
        """
        try:
            response_data: Final = raw_response.json()
        except Exception as e:
            raise SagemakerError(
                message=f"Failed to parse response: {e}",
                status_code=raw_response.status_code,
            )

        # Handle both raw array format (TEI) and wrapped format (standard HF)
        if isinstance(response_data, list):
            # TEI and some HF models return raw embedding arrays directly
            embeddings = response_data
        elif isinstance(response_data, dict) and "embedding" in response_data:
            # Standard HF format with "embedding" key
            embeddings = response_data["embedding"]
        else:
            raise SagemakerError(
                status_code=500,
                message=f"Unexpected response format. Expected list or dict with 'embedding' key, got: {type(response_data).__name__}",
            )

        if not isinstance(embeddings, list):
            raise SagemakerError(
                status_code=422,
                message=f"HF response not in expected format - {embeddings}",
            )

        output_data: Final = []
        for idx, embedding in enumerate(embeddings):
            output_data.append({"object": "embedding", "index": idx, "embedding": embedding})

        model_response.object = "list"
        model_response.data = output_data
        model_response.model = model

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Inspect the type named in the message, then log the raw body to see the actual key structure.
2. If the key differs (e.g. 'embeddings'), change the container to return either a bare list or {'embedding': [[...], ...]}.
3. For TEI deployments, confirm you are hitting the /embed route and not another route with a different response shape.

Example fix

# custom container - before
return {'embeddings': vectors}
# after (schema LiteLLM understands)
return {'embedding': vectors}

from litellm import SagemakerError

try:
    resp = litellm.embedding(model='sagemaker/hf-emb', input=texts)
except SagemakerError as e:
    if 'Unexpected response format' in str(e):
        log.error('embedding schema changed: %s', e.message)
    raise

When it happens

Trigger: The endpoint returns JSON like {'embeddings': [...]}, {'error': {...}} with HTTP 200, or a scalar/string body; i.e. a schema the transformer never anticipated.

Common situations: Custom HF inference containers returning their own key name; newer TEI versions changing the response envelope; models that wrap results in extra metadata objects.

Related errors

• Failed to parse response: {e}
• HF response not in expected format - {embeddings}
• No embedding data found in response: {response}
• LiteLLM Error: Unable to parse sagemaker RAW RESPONSE {json.
• Input must be a list of strings

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/89b0dab9e2af61e9. Report an issue: GitHub.