{"record":{"id":"a6bb7398756759ec","repo":"BerriAI/litellm","slug":"embedding-response-does-not-contain-predictions","errorCode":null,"errorMessage":"embedding response does not contain 'predictions', got {_json_response}","messagePattern":"embedding response does not contain 'predictions', got (.+?)","errorType":"exception","errorClass":"InternalServerError","httpStatus":500,"severity":"error","filePath":"litellm/llms/vertex_ai/multimodal_embeddings/transformation.py","lineNumber":213,"sourceCode":"        return cast(dict, request_data)\n\n    def transform_embedding_response(\n        self,\n        model: str,\n        raw_response: Response,\n        model_response: EmbeddingResponse,\n        logging_obj: LiteLLMLoggingObj,\n        api_key: str | None,\n        request_data: dict,\n        optional_params: dict,\n        litellm_params: dict,\n    ) -> EmbeddingResponse:\n        if raw_response.status_code != 200:\n            raise Exception(f\"Error: {raw_response.status_code} {raw_response.text}\")\n\n        _json_response: Final = raw_response.json()\n        if \"predictions\" not in _json_response:\n            raise InternalServerError(\n                message=f\"embedding response does not contain 'predictions', got {_json_response}\",\n                llm_provider=\"vertex_ai\",\n                model=model,\n            )\n        _predictions: Final = _json_response[\"predictions\"]\n        vertex_predictions: Final = MultimodalPredictions(predictions=_predictions)\n        model_response.data = self.transform_embedding_response_to_openai(predictions=vertex_predictions)\n        model_response.model = model\n\n        model_response.usage = self.calculate_usage(\n            request_data=cast(VertexMultimodalEmbeddingRequest, request_data),\n            vertex_predictions=vertex_predictions,\n        )\n\n        return model_response\n\n    def calculate_usage(\n        self,","sourceCodeStart":195,"sourceCodeEnd":231,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/llms/vertex_ai/multimodal_embeddings/transformation.py#L195-L231","documentation":"Raised as litellm InternalServerError when the multimodal embeddings endpoint answers HTTP 200 but the JSON body has no 'predictions' key. LiteLLM expects the Vertex predict shape ({'predictions': [...]}) and treats any other 200 body as a provider-side failure. The full unexpected body is embedded in the message for diagnosis.","triggerScenarios":"Vertex returns 200 with an error or status object instead of predictions (partial outage), a proxy or custom api_base rewrites the response into a different envelope, or the model name routes to a non-multimodal endpoint whose response schema differs.","commonSituations":"Pointing api_base at a gateway that wraps responses in its own envelope; using a model id not served by the multimodal embeddings endpoint; transient GCP incidents returning 200 with empty or partial bodies.","solutions":["Inspect the JSON body printed in the message to see exactly what Vertex returned instead of predictions","Confirm the model is a Vertex multimodal embedding model and the URL targets the multimodal predict endpoint","If behind a proxy, make sure it forwards the raw Vertex response unchanged","Treat as transient if the body looks like a partial/empty GCP response: retry, and check the Google Cloud status dashboard"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"from litellm.exceptions import InternalServerError\n\ntry:\n    resp = litellm.embedding(model=model, input=inputs)\nexcept InternalServerError as e:\n    if \"does not contain 'predictions'\" in str(e):\n        log.warning('vertex returned 200 without predictions; retrying once')\n        resp = litellm.embedding(model=model, input=inputs)\n    else:\n        raise","preventionTips":["Do not wrap Vertex endpoints in proxies that change the response envelope","Pin the exact multimodal model id supported by the endpoint","Log raw responses in staging to catch schema drift early"],"tags":["vertex-ai","embeddings","response-schema","internal-server-error"],"backgroundTag":"unexpected-response-schema","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}