BerriAI/litellm · error · ValueError

Unable to pick client for private endpoint

Error message

Unable to pick client for private endpoint

What it means

Generic sentinel in the private-endpoint branch: no client-selection condition matched (neither the sync gapic client nor another configured path could be chosen), so LiteLLM cannot build a prediction client for the request.

Source

Thrown at litellm/llms/vertex_ai/vertex_ai_non_gemini.py:357

            request_str += f"llm_model = aiplatform.gapic.PredictionServiceClient(client_options={client_options}, credentials=...)\n"
            endpoint_path = llm_model.endpoint_path(project=vertex_project, location=vertex_location, endpoint=model)
            request_str += f"llm_model.predict(endpoint={endpoint_path}, instances={instances})\n"
            response = llm_model.predict(endpoint=endpoint_path, instances=instances).predictions

            completion_response = response[0]
            if isinstance(completion_response, str) and "\nOutput:\n" in completion_response:
                completion_response = completion_response.split("\nOutput:\n", 1)[1]
            if stream is True:
                response = TextStreamer(completion_response)
                return response
        elif mode == "private":
            """
            Vertex AI Model Garden deployed on private endpoint
            """
            if instances is None:
                raise ValueError("instances are required for private endpoint")
            if llm_model is None:
                raise ValueError("Unable to pick client for private endpoint")
            ## LOGGING
            logging_obj.pre_call(
                input=prompt,
                api_key=None,
                additional_args={
                    "complete_input_dict": optional_params,
                    "request_str": request_str,
                },
            )
            request_str += f"llm_model.predict(instances={instances})\n"
            response = llm_model.predict(instances=instances).predictions

            completion_response = response[0]
            if isinstance(completion_response, str) and "\nOutput:\n" in completion_response:
                completion_response = completion_response.split("\nOutput:\n", 1)[1]
            if stream is True:
                response = TextStreamer(completion_response)
                return response

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Verify the private endpoint configuration (endpoint_id, project, location) so a prediction client can be constructed.
2. Confirm google-cloud-aiplatform is installed and credentials are valid for the endpoint's project.

When it happens

Trigger: Thrown at litellm/llms/vertex_ai/vertex_ai_non_gemini.py:357 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/12edb6481bf37fc4. Report an issue: GitHub.