BerriAI/litellm · error · InternalServerError

embedding response does not contain 'predictions', got {_jso

Error message

embedding response does not contain 'predictions', got {_json_response}

What it means

Raised as litellm InternalServerError when the multimodal embeddings endpoint answers HTTP 200 but the JSON body has no 'predictions' key. LiteLLM expects the Vertex predict shape ({'predictions': [...]}) and treats any other 200 body as a provider-side failure. The full unexpected body is embedded in the message for diagnosis.

Source

Thrown at litellm/llms/vertex_ai/multimodal_embeddings/transformation.py:213

        return cast(dict, request_data)

    def transform_embedding_response(
        self,
        model: str,
        raw_response: Response,
        model_response: EmbeddingResponse,
        logging_obj: LiteLLMLoggingObj,
        api_key: str | None,
        request_data: dict,
        optional_params: dict,
        litellm_params: dict,
    ) -> EmbeddingResponse:
        if raw_response.status_code != 200:
            raise Exception(f"Error: {raw_response.status_code} {raw_response.text}")

        _json_response: Final = raw_response.json()
        if "predictions" not in _json_response:
            raise InternalServerError(
                message=f"embedding response does not contain 'predictions', got {_json_response}",
                llm_provider="vertex_ai",
                model=model,
            )
        _predictions: Final = _json_response["predictions"]
        vertex_predictions: Final = MultimodalPredictions(predictions=_predictions)
        model_response.data = self.transform_embedding_response_to_openai(predictions=vertex_predictions)
        model_response.model = model

        model_response.usage = self.calculate_usage(
            request_data=cast(VertexMultimodalEmbeddingRequest, request_data),
            vertex_predictions=vertex_predictions,
        )

        return model_response

    def calculate_usage(
        self,

View on GitHub (pinned to 77b7c6c40c)

Solutions

Inspect the JSON body printed in the message to see exactly what Vertex returned instead of predictions
Confirm the model is a Vertex multimodal embedding model and the URL targets the multimodal predict endpoint
If behind a proxy, make sure it forwards the raw Vertex response unchanged
Treat as transient if the body looks like a partial/empty GCP response: retry, and check the Google Cloud status dashboard

Defensive patterns

Strategy: retry

Try / catch

from litellm.exceptions import InternalServerError

try:
    resp = litellm.embedding(model=model, input=inputs)
except InternalServerError as e:
    if "does not contain 'predictions'" in str(e):
        log.warning('vertex returned 200 without predictions; retrying once')
        resp = litellm.embedding(model=model, input=inputs)
    else:
        raise

Prevention

Do not wrap Vertex endpoints in proxies that change the response envelope
Pin the exact multimodal model id supported by the endpoint
Log raw responses in staging to catch schema drift early

When it happens

Trigger: Vertex returns 200 with an error or status object instead of predictions (partial outage), a proxy or custom api_base rewrites the response into a different envelope, or the model name routes to a non-multimodal endpoint whose response schema differs.

Common situations: Pointing api_base at a gateway that wraps responses in its own envelope; using a model id not served by the multimodal embeddings endpoint; transient GCP incidents returning 200 with empty or partial bodies.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/a6bb7398756759ec. Report an issue: GitHub.