apache/beam · warning

A non-standard version of Beam SDK detected

Error message

A non-standard version of Beam SDK detected: %s. Dataflow runner will use container image tag %s. This use case is not supported.

What it means

apache_beam emits this UserWarning when the installed Beam SDK version string is not a plain 'base' version (e.g. a dev build like '2.53.0.dev0' or a version with local suffix). The Dataflow runner normalizes it with packaging.version's base_version and will launch workers with the container image tagged with that base version, but this combination is unsupported, so it warns.

Solutions

1. Install an official release from PyPI: pip install apache-beam[<extras>]==<stable version> and rerun the Dataflow job.
2. If you must use a dev build, verify the resulting container image tag (base_version) exists on gcr.io/cloud-dataflow and pin matching runner/container versions.
3. Set a supported container image explicitly via pipeline_options.view_as(SetupOptions).sdk_container_image to avoid the default tag choice.
4. If the warning is acceptable (local experiments only), suppress it with warnings.filterwarnings('ignore', message='A non-standard version of Beam SDK detected').

Example fix

# before
pip install -e apache-beam  # dev build '2.61.0.dev0'
# after
pip install apache-beam==2.61.0

import apache_beam as beam
from packaging.version import Version
v = Version(beam.__version__)
if v.base_version != beam.__version__:
    raise RuntimeError(f"Non-standard Beam version {beam.__version__}; install an official release for Dataflow")

def is_official_beam_version(version_str):
    from packaging.version import Version
    return Version(version_str).base_version == version_str

When it happens

Trigger: Running a Dataflow pipeline from an SDK whose beam_version.__version__ differs from version.parse(__version__).base_version — e.g. installing from a git checkout, a nightly wheel, or a locally built/patched version string like '2.61.0.dev' or '2.61.0+local'.

Common situations: Developers installing Beam from source (git clone + pip install -e .), CI jobs using nightly snapshots, or locally patched SDK builds submitting jobs to real Dataflow instead of DirectRunner.

Related errors

• Unsupported environment
• A cannot be expanded
• bad I
• bad KV
• bad N

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e205745971488e72. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/dataflow/internal/apiclient.py:1202

  dist_update_proto.update({"sum": distribution_update.sum})
  # DataflowDistributionCounter needs to translate histogram
  if isinstance(distribution_update, DataflowDistributionCounter):
    histogram = Histogram()
    distribution_update.translate_to_histogram(histogram)
    dist_update_proto.update({"firstBucketOffset": histogram.firstBucketOffset})
    dist_update_proto.update({"bucketCounts": histogram.bucketCounts})
  metric_update_proto.distribution = dist_update_proto


# TODO: Used in legacy batch worker. Delete after Dataflow Portable Runner transition.
def translate_value(value, metric_update_proto: dataflow.MetricUpdate):
  metric_update_proto.scalar = value


def _get_container_image_tag():
  base_version = version.parse(beam_version.__version__).base_version
  if base_version != beam_version.__version__:
    warnings.warn(
        "A non-standard version of Beam SDK detected: %s. "
        "Dataflow runner will use container image tag %s. "
        "This use case is not supported." %
        (beam_version.__version__, base_version))
  return base_version


def get_container_image_from_options(pipeline_options):
  """For internal use only; no backwards-compatibility guarantees.

    Args:
      pipeline_options (PipelineOptions): A container for pipeline options.

    Returns:
      str: Container image for remote execution.
  """
  worker_options = pipeline_options.view_as(WorkerOptions)
  if worker_options.sdk_container_image:

View on GitHub (pinned to 12126d8942)