apache/beam · critical · RuntimeError

Pipeline construction environment and pipeline runtime…

Error message

Pipeline construction environment and pipeline runtime environment are not compatible. If you use a custom container image, check that the Python interpreter minor version and the Apache Beam version in your image match the versions used at pipeline construction time. Submission environment: {c}. Runtime environment: {runtime_sdk}.

What it means

Before executing a process bundle, the Fn API worker verifies that every transform's environment capabilities (SDK version capability) are compatible with the running SDK. If the pipeline was constructed with a different Beam version or Python minor version than the worker's runtime image, it raises this RuntimeError. This almost always means the custom container image does not match the pipeline construction environment.

Solutions

Rebuild the custom container image with the same Python minor version and Apache Beam version used at pipeline construction
Pin apache_beam to the same version in both submission and runtime environments (--sdk_location / matching requirements)
Re-generate the pipeline after upgrading the runtime environment
Verify with 'python --version' and 'pip show apache-beam' in both environments and align them

Example fix

# before: submitted with apache-beam==2.50.0 on py3.11, image built with apache-beam==2.46.0 on py3.9
// after: Dockerfile
FROM apache/beam_python3.11_sdk:2.50.0
# same Beam version and Python minor version as the submitting environment

Defensive patterns

Strategy: validation

Validate before calling

import apache_beam as beam, subprocess, sys
sub_ver = beam.version.__version__
sub_py = f'{sys.version_info.major}.{sys.version_info.minor}'
# check the container image you submit matches:
# docker run --rm my-image python -c "import apache_beam as b,sys;print(b.version.__version__, f'{sys.version_info.major}.{sys.version_info.minor}')"
assert sub_ver == image_ver and sub_py == image_py, 'Runtime environment mismatch'

Try / catch

try:
    pipeline.run().wait_until_finish()
except RuntimeError as e:
    if 'Pipeline construction environment and pipeline runtime environment are not compatible' in str(e):
        logging.error('Rebuild the worker image with apache-beam==%s on Python %s', sub_ver, sub_py)
        raise
    raise

Prevention

Base custom images on apache/beam_python<X.Y>_sdk:<same-version-as-submit-env>
Pin apache_beam version in requirements and Dockerfile together
Rebuild images whenever the submission environment's Beam version changes
Print both environments' versions in CI before submitting jobs

When it happens

Trigger: Running a portable/Dataflow/Flink job where process_bundle_descriptor environments carry an SDK_VERSION capability incompatible with the worker's runtime_sdk (checked in _verify_descriptor_created_in_a_compatible_env during bundle_processor init).

Common situations: Custom container image with a different Python minor version (e.g. built on 3.9, submitted from 3.11); upgrading apache_beam locally without rebuilding the worker image; staging an old pipeline proto against new workers.

Understand the failure class

Background: "environment variable is not set" and "Missing keys in environment" errors: what missing required env var messages mean and how to fix them — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b3c7dcd0616c2e51. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/worker/bundle_processor.py:1081

  if submission == runtime:
    return True
  if 'rc' in submission and runtime in submission:
    # TODO(https://github.com/apache/beam/issues/28084): Loosen
    # the check for RCs until RC containers install the matching version.
    return True
  return False


def _verify_descriptor_created_in_a_compatible_env(
    process_bundle_descriptor: beam_fn_api_pb2.ProcessBundleDescriptor) -> None:

  runtime_sdk = environments.sdk_base_version_capability()
  for t in process_bundle_descriptor.transforms.values():
    env = process_bundle_descriptor.environments[t.environment_id]
    for c in env.capabilities:
      if (c.startswith(environments.SDK_VERSION_CAPABILITY_PREFIX) and
          not _environments_compatible(c, runtime_sdk)):
        raise RuntimeError(
            "Pipeline construction environment and pipeline runtime "
            "environment are not compatible. If you use a custom "
            "container image, check that the Python interpreter minor version "
            "and the Apache Beam version in your image match the versions "
            "used at pipeline construction time. "
            f"Submission environment: {c}. "
            f"Runtime environment: {runtime_sdk}.")

  # TODO: Consider warning on mismatches in versions of installed packages.


class BundleProcessor(object):
  """ A class for processing bundles of elements. """
  def __init__(
      self,
      runner_capabilities: frozenset[str],
      process_bundle_descriptor: beam_fn_api_pb2.ProcessBundleDescriptor,
      state_handler: sdk_worker.CachingStateHandler,

View on GitHub (pinned to 12126d8942)