apache/beam · error · ValueError

Failed to contact endpoint

Error message

Failed to contact endpoint %s, got exception: %s

What it means

_retrieve_endpoint verifies connectivity by calling endpoint.list_models(); any exception there is wrapped as ValueError('Failed to contact endpoint %s, got exception: %s', ...). Note the format-string args are passed as separate parameters (not %-formatted), so the displayed message can be misleading, but the intent is that the Vertex AI endpoint could not be contacted.

Solutions

1. Verify endpoint_id and location (project/region must match where the endpoint is deployed)
2. Grant the caller the aiplatform.endpoints.get/list permission (Vertex AI User role)
3. Check network reachability (VPC peering / Private Google Access for private endpoints) and retry on transient failures
4. Inspect the underlying exception (second arg of the raised ValueError) for the real cause

Example fix

// before
handler = VertexAIModelHandlerGPU(endpoint_id='123', location='us-east1', ...)
// after  # endpoint actually lives in us-central1
handler = VertexAIModelHandlerGPU(endpoint_id='123', location='us-central1', ...)

# preflight check before building the pipeline
from google.cloud import aiplatform
aiplatform.init(project=project, location=location)
endpoint = aiplatform.Endpoint(endpoint_name=endpoint_id)
models = endpoint.list_models()  # raises early if unreachable

try:
    handler = VertexAIModelHandlerGPU(endpoint_id=ep, project=p, location=l)
except ValueError as e:
    logger.error('Endpoint contact failed (check id/region/permissions/VPC): %s', e)
    raise

When it happens

Trigger: Constructing VertexAIModelHandlerGPU or calling create_client when endpoint.list_models() fails: wrong endpoint id, wrong region, missing AI Platform permissions (aiplatform.endpoints.get), or no network path (e.g. private endpoint without VPC access).

Common situations: Typos in endpoint_id or location; service account lacking Vertex AI permissions; running on-prem/without VPC peering for a private endpoint; transient GCP API outages.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

• A VPC network must be provided to use a private endpoint.
• Endpoint has no models deployed to it.
• Failed to retrieve secret bytes for secret
• Unable to import VertexAIModelHandlerJSON. Please install…
• Vertex AI Feature Store

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d7dfedec5a714e0c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/ml/inference/vertex_ai_inference.py:196

        endpoint
    Returns:
      An aiplatform.Endpoint object
    Raises:
      ValueError: if endpoint is inactive or has no models deployed to it.
    """
    if is_private:
      endpoint: aiplatform.Endpoint = aiplatform.PrivateEndpoint(
          endpoint_name=endpoint_id, location=location)
      LOGGER.debug("Treating endpoint %s as private", endpoint_id)
    else:
      endpoint = aiplatform.Endpoint(
          endpoint_name=endpoint_id, location=location)
      LOGGER.debug("Treating endpoint %s as public", endpoint_id)

    try:
      mod_list = endpoint.list_models()
    except Exception as e:
      raise ValueError(
          "Failed to contact endpoint %s, got exception: %s", endpoint_id, e)

    if len(mod_list) == 0:
      raise ValueError("Endpoint %s has no models deployed to it.", endpoint_id)

    return endpoint

  def create_client(self) -> aiplatform.Endpoint:
    """Loads the Endpoint object used to build and send prediction request to
    Vertex AI.
    """
    # Check to make sure the endpoint is still active since pipeline
    # construction time
    ep = self._retrieve_endpoint(
        self.endpoint_name, self.location, self.is_private)
    return ep

  def request(

View on GitHub (pinned to 12126d8942)