BerriAI/litellm · error · ValueError

instances are required for private endpoint

Error message

instances are required for private endpoint

What it means

Sync private-endpoint guard: mode == 'private' but the caller passed instances=None. The PredictionServiceClient.predict call needs instance payloads, so the request is rejected before contacting the endpoint.

Source

Thrown at litellm/llms/vertex_ai/vertex_ai_non_gemini.py:355

                credentials=creds,
            )
            request_str += f"llm_model = aiplatform.gapic.PredictionServiceClient(client_options={client_options}, credentials=...)\n"
            endpoint_path = llm_model.endpoint_path(project=vertex_project, location=vertex_location, endpoint=model)
            request_str += f"llm_model.predict(endpoint={endpoint_path}, instances={instances})\n"
            response = llm_model.predict(endpoint=endpoint_path, instances=instances).predictions

            completion_response = response[0]
            if isinstance(completion_response, str) and "\nOutput:\n" in completion_response:
                completion_response = completion_response.split("\nOutput:\n", 1)[1]
            if stream is True:
                response = TextStreamer(completion_response)
                return response
        elif mode == "private":
            """
            Vertex AI Model Garden deployed on private endpoint
            """
            if instances is None:
                raise ValueError("instances are required for private endpoint")
            if llm_model is None:
                raise ValueError("Unable to pick client for private endpoint")
            ## LOGGING
            logging_obj.pre_call(
                input=prompt,
                api_key=None,
                additional_args={
                    "complete_input_dict": optional_params,
                    "request_str": request_str,
                },
            )
            request_str += f"llm_model.predict(instances={instances})\n"
            response = llm_model.predict(instances=instances).predictions

            completion_response = response[0]
            if isinstance(completion_response, str) and "\nOutput:\n" in completion_response:
                completion_response = completion_response.split("\nOutput:\n", 1)[1]
            if stream is True:

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Provide the 'instances' payload required by the private endpoint in the request body.
2. Check the endpoint's expected input schema and format instances accordingly.

When it happens

Trigger: Thrown at litellm/llms/vertex_ai/vertex_ai_non_gemini.py:355 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/d2c9d0c3df945c89. Report an issue: GitHub.