apache/beam · error · ValueError

Unable to import VertexAIModelHandlerJSON. Please install gc

Error message

Unable to import VertexAIModelHandlerJSON. Please install gcp dependencies: `pip install apache_beam[gcp]`

What it means

Raised when the Vertex AI handler class VertexAIModelHandlerJSON cannot be imported because the GCP extra dependencies of apache_beam are not installed. yaml_ml imports the handler lazily inside __init__ and converts the ImportError into a ValueError with install instructions.

Source

Thrown at sdks/python/apache_beam/yaml/yaml_ml.py:258

      invoke_route: The custom route path to use when invoking
        endpoints with arbitrary prediction routes. When specified, uses
        `Endpoint.invoke()` instead of `Endpoint.predict()`. The route
        should start with a forward slash, e.g., "/predict/v1".
        See
        https://cloud.google.com/vertex-ai/docs/predictions/use-arbitrary-custom-routes
        for more information.
      min_batch_size: The minimum batch size to use when batching
        inputs.
      max_batch_size: The maximum batch size to use when batching
        inputs.
      max_batch_duration_secs: The maximum amount of time to buffer
        a batch before emitting; used in streaming contexts.
    """

    try:
      from apache_beam.ml.inference.vertex_ai_inference import VertexAIModelHandlerJSON
    except ImportError:
      raise ValueError(
          'Unable to import VertexAIModelHandlerJSON. Please '
          'install gcp dependencies: `pip install apache_beam[gcp]`')

    _handler = VertexAIModelHandlerJSON(
        endpoint_id=str(endpoint_id),
        project=project,
        location=location,
        experiment=experiment,
        network=network,
        private=private,
        invoke_route=invoke_route,
        min_batch_size=min_batch_size,
        max_batch_size=max_batch_size,
        max_batch_duration_secs=max_batch_duration_secs)

    super().__init__(_handler, preprocess, postprocess)

  @staticmethod

View on GitHub (pinned to 12126d8942)

Solutions

1. Install GCP extras: pip install apache_beam[gcp]
2. Ensure the runtime (Dataflow container, cluster worker, Flink/Kubernetes image) has the same extras installed
3. If you don't need Vertex AI, switch the model handler type to a local or other supported handler

Example fix

# before
pip install apache_beam
# after
pip install 'apache_beam[gcp]'

try:
    from apache_beam.ml.inference.vertex_ai_inference import VertexAIModelHandlerJSON
except ImportError:
    raise SystemExit("Run: pip install 'apache_beam[gcp]'")

def has_vertex_ai_handler():
    import importlib.util
    return importlib.util.find_spec('apache_beam.ml.inference.vertex_ai_inference') is not None

try:
    transform = RunInference(model_handler=handler_spec)
except ValueError as e:
    if 'install gcp dependencies' in str(e):
        install_gcp_extras()
    else:
        raise

When it happens

Trigger: Using the YAML RunInference ML transform with a Vertex AI model handler (type VertexAI / vertex_ai_model_handler_json) on an environment where apache_beam was installed without the [gcp] extra, so google-cloud-aiplatform and related packages are absent.

Common situations: Deploying a Beam YAML pipeline to an environment with only the base apache_beam package; using --extra-package or requirements files that omit gcp extras; running in containers built from plain apache_beam images.

Understand the failure class

Background: "X is not installed. Please install it with pip install Y": missing optional dependency errors — ImportError/ValueError raised when a library's optional extra was never installed — this error's family across 22 libraries.

Related errors

• Vertex AI Feature Store %s does not exists in %s
• google-cloud-bigquery-storage is required for ReadBigQueryCh
• pyarrow is required for ReadBigQueryChangeHistory. Install i
• query.project cannot be empty
• query cannot be empty

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/78c72cf96477a894. Report an issue: GitHub.