{"record":{"id":"335e04fe80ac08df","repo":"apache/beam","slug":"incompatible-environments-s-s","errorCode":null,"errorMessage":"Incompatible environments: '%s' != '%s'","messagePattern":"Incompatible environments: '(.+?)' != '(.+?)'","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/portability/fn_api_runner/translations.py","lineNumber":183,"sourceCode":"        must_follow,\n        downstream_side_inputs)\n\n  @staticmethod\n  def _extract_environment(transform):\n    # type: (beam_runner_api_pb2.PTransform) -> Optional[str]\n    environment = transform.environment_id\n    return environment if environment else None\n\n  @staticmethod\n  def _merge_environments(env1, env2):\n    # type: (Optional[str], Optional[str]) -> Optional[str]\n    if env1 is None:\n      return env2\n    elif env2 is None:\n      return env1\n    else:\n      if env1 != env2:\n        raise ValueError(\n            \"Incompatible environments: '%s' != '%s'\" %\n            (str(env1).replace('\\n', ' '), str(env2).replace('\\n', ' ')))\n      return env1\n\n  def can_fuse(self, consumer, context):\n    # type: (Stage, TransformContext) -> bool\n    try:\n      self._merge_environments(self.environment, consumer.environment)\n    except ValueError:\n      return False\n\n    def no_overlap(a, b):\n      return not a or not b or not a.intersection(b)\n\n    return (\n        not consumer.forced_root and not self in consumer.must_follow and\n        self.is_all_sdk_urns(context) and consumer.is_all_sdk_urns(context) and\n        no_overlap(self.downstream_side_inputs, consumer.side_inputs()))","sourceCodeStart":165,"sourceCodeEnd":201,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/portability/fn_api_runner/translations.py#L165-L201","documentation":"During stage fusion, translations._merge_environments checks whether two adjacent stages use identical execution environments. If both stages have environments and they differ (SDK version, container image, dependencies), fusion is refused with this ValueError because the merged stage could not run in a single worker environment.","triggerScenarios":"Building an optimized pipeline where a consumer transform declares a different environment than its producer — e.g. different docker container_image, different SDK harness version, or one transform with custom setup.py dependencies.","commonSituations":"Mixing transforms from different SDK versions in one pipeline, custom container images applied to only part of a pipeline, or external pipeline fragments (e.g. cross-language transforms) with their own environment merged into a pipeline.","solutions":["Unify environments: use the same container_image/SDK version for the transforms you want fused","Apply custom environments consistently to the whole pipeline rather than to individual transforms","If differing environments are intentional, accept non-fused stages — remove or adjust the fusion optimization, or insert an explicit boundary","Compare the two environment protos in the error message (newline-flattened) to see exactly which field differs"],"exampleFix":"// before: custom env on one transform only\nPCollection<int> out = transform.setInput(p).setEnvironment(customEnv).expand();\n// after: same env pipeline-wide (or none, letting the default apply)\nPCollection<int> out = transform.setInput(p).expand();","handlingStrategy":"validation","validationCode":"envs = {t.spec.environment_id for t in pipeline.components.transforms.values() if t.spec.environment_id}\nif len(envs) > 1:\n    print('Differing environments:', [pipeline.components.protos.environments[e] for e in envs])","typeGuard":null,"tryCatchPattern":"try:\n    optimized = pipeline.run()\nexcept ValueError as e:\n    if 'Incompatible environments' in str(e):\n        log.error('Stage fusion refused: %s', e)  # envs differ; unify container_image/deps\n    raise","preventionTips":["Use one container_image/SDK version across the whole pipeline","Apply custom environments pipeline-wide, not per-transform","Compare env protos in the error text to identify the differing field","Fully expand cross-language transforms and align their environments before optimization"],"tags":["python","apache-beam","stage-fusion","environment"],"backgroundTag":"incompatible-environment","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}