{"record":{"id":"b3c7dcd0616c2e51","repo":"apache/beam","slug":"pipeline-construction-environment-and-pipeline-runtime","errorCode":null,"errorMessage":"Pipeline construction environment and pipeline runtime environment are not compatible. If you use a custom container image, check that the Python interpreter minor version and the Apache Beam version in your image match the versions used at pipeline construction time. Submission environment: {c}. Runtime environment: {runtime_sdk}.","messagePattern":"Pipeline construction environment and pipeline runtime environment are not compatible\\. If you use a custom container image, check that the Python interpreter minor version and the Apache Beam version in your image match the versions used at pipeline construction time\\. Submission environment: (.+?)\\. Runtime environment: (.+?)\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"sdks/python/apache_beam/runners/worker/bundle_processor.py","lineNumber":1081,"sourceCode":"  if submission == runtime:\n    return True\n  if 'rc' in submission and runtime in submission:\n    # TODO(https://github.com/apache/beam/issues/28084): Loosen\n    # the check for RCs until RC containers install the matching version.\n    return True\n  return False\n\n\ndef _verify_descriptor_created_in_a_compatible_env(\n    process_bundle_descriptor: beam_fn_api_pb2.ProcessBundleDescriptor) -> None:\n\n  runtime_sdk = environments.sdk_base_version_capability()\n  for t in process_bundle_descriptor.transforms.values():\n    env = process_bundle_descriptor.environments[t.environment_id]\n    for c in env.capabilities:\n      if (c.startswith(environments.SDK_VERSION_CAPABILITY_PREFIX) and\n          not _environments_compatible(c, runtime_sdk)):\n        raise RuntimeError(\n            \"Pipeline construction environment and pipeline runtime \"\n            \"environment are not compatible. If you use a custom \"\n            \"container image, check that the Python interpreter minor version \"\n            \"and the Apache Beam version in your image match the versions \"\n            \"used at pipeline construction time. \"\n            f\"Submission environment: {c}. \"\n            f\"Runtime environment: {runtime_sdk}.\")\n\n  # TODO: Consider warning on mismatches in versions of installed packages.\n\n\nclass BundleProcessor(object):\n  \"\"\" A class for processing bundles of elements. \"\"\"\n  def __init__(\n      self,\n      runner_capabilities: frozenset[str],\n      process_bundle_descriptor: beam_fn_api_pb2.ProcessBundleDescriptor,\n      state_handler: sdk_worker.CachingStateHandler,","sourceCodeStart":1063,"sourceCodeEnd":1099,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/worker/bundle_processor.py#L1063-L1099","documentation":"Before executing a process bundle, the Fn API worker verifies that every transform's environment capabilities (SDK version capability) are compatible with the running SDK. If the pipeline was constructed with a different Beam version or Python minor version than the worker's runtime image, it raises this RuntimeError. This almost always means the custom container image does not match the pipeline construction environment.","triggerScenarios":"Running a portable/Dataflow/Flink job where process_bundle_descriptor environments carry an SDK_VERSION capability incompatible with the worker's runtime_sdk (checked in _verify_descriptor_created_in_a_compatible_env during bundle_processor init).","commonSituations":"Custom container image with a different Python minor version (e.g. built on 3.9, submitted from 3.11); upgrading apache_beam locally without rebuilding the worker image; staging an old pipeline proto against new workers.","solutions":["Rebuild the custom container image with the same Python minor version and Apache Beam version used at pipeline construction","Pin apache_beam to the same version in both submission and runtime environments (--sdk_location / matching requirements)","Re-generate the pipeline after upgrading the runtime environment","Verify with 'python --version' and 'pip show apache-beam' in both environments and align them"],"exampleFix":"# before: submitted with apache-beam==2.50.0 on py3.11, image built with apache-beam==2.46.0 on py3.9\n// after: Dockerfile\nFROM apache/beam_python3.11_sdk:2.50.0\n# same Beam version and Python minor version as the submitting environment","handlingStrategy":"validation","validationCode":"import apache_beam as beam, subprocess, sys\nsub_ver = beam.version.__version__\nsub_py = f'{sys.version_info.major}.{sys.version_info.minor}'\n# check the container image you submit matches:\n# docker run --rm my-image python -c \"import apache_beam as b,sys;print(b.version.__version__, f'{sys.version_info.major}.{sys.version_info.minor}')\"\nassert sub_ver == image_ver and sub_py == image_py, 'Runtime environment mismatch'","typeGuard":null,"tryCatchPattern":"try:\n    pipeline.run().wait_until_finish()\nexcept RuntimeError as e:\n    if 'Pipeline construction environment and pipeline runtime environment are not compatible' in str(e):\n        logging.error('Rebuild the worker image with apache-beam==%s on Python %s', sub_ver, sub_py)\n        raise\n    raise","preventionTips":["Base custom images on apache/beam_python<X.Y>_sdk:<same-version-as-submit-env>","Pin apache_beam version in requirements and Dockerfile together","Rebuild images whenever the submission environment's Beam version changes","Print both environments' versions in CI before submitting jobs"],"tags":["apache-beam","python","environment-mismatch","container-image"],"backgroundTag":"missing-env-var","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}