BerriAI/litellm · error · SagemakerError

HF response not in expected format - {embeddings}

Error message

HF response not in expected format - {embeddings}

What it means

Once the embeddings payload is extracted (raw list or dict['embedding']), LiteLLM requires it to be a JSON list so it can iterate per-input vectors. If the value under 'embedding' (or the whole payload) is an object, string, or number, it raises SagemakerError 422 echoing the malformed value.

Source

Thrown at litellm/llms/sagemaker/embedding/transformation.py:118

                message=f"Failed to parse response: {e}",
                status_code=raw_response.status_code,
            )

        # Handle both raw array format (TEI) and wrapped format (standard HF)
        if isinstance(response_data, list):
            # TEI and some HF models return raw embedding arrays directly
            embeddings = response_data
        elif isinstance(response_data, dict) and "embedding" in response_data:
            # Standard HF format with "embedding" key
            embeddings = response_data["embedding"]
        else:
            raise SagemakerError(
                status_code=500,
                message=f"Unexpected response format. Expected list or dict with 'embedding' key, got: {type(response_data).__name__}",
            )

        if not isinstance(embeddings, list):
            raise SagemakerError(
                status_code=422,
                message=f"HF response not in expected format - {embeddings}",
            )

        output_data: Final = []
        for idx, embedding in enumerate(embeddings):
            output_data.append({"object": "embedding", "index": idx, "embedding": embedding})

        model_response.object = "list"
        model_response.data = output_data
        model_response.model = model

        # Calculate usage from request data
        input_texts: Final = request_data.get("inputs", [])
        input_tokens = 0
        for text in input_texts:
            input_tokens += len(text.split())  # Simple word count fallback

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Log the value shown in the error message to see the exact malformed shape.
2. Make the container return a list of lists even for a single input: {'embedding': [[...]]}.
3. Keep single-request and batch-request response shapes identical in custom inference code.

Example fix

# custom container - before (single input)
return {'embedding': [0.1, 0.2, 0.3]}
# after (list of per-input vectors)
return {'embedding': [[0.1, 0.2, 0.3]]}

from litellm import SagemakerError

try:
    resp = litellm.embedding(model='sagemaker/hf-emb', input=texts)
except SagemakerError as e:
    if e.status_code == 422 and 'not in expected format' in str(e):
        log.error('malformed embedding payload: %s', e.message)
    raise

When it happens

Trigger: A response like {'embedding': {'vector': [...]}} or {'embedding': 'error message'} - the key exists but does not map to an array of embedding arrays.

Common situations: Containers that return a single vector object for single-input requests; error payloads nested under the embedding key; hand-written inference scripts with inconsistent shapes between single and batch requests.

Related errors

• Failed to parse response: {e}
• Unexpected response format. Expected list or dict with 'embe
• No embedding data found in response: {response}
• LiteLLM Error: Unable to parse sagemaker RAW RESPONSE {json.
• Input must be a list of strings

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/a29c1a9f19b4077b. Report an issue: GitHub.