{"record":{"id":"e6c22f9b534a1568","repo":"apache/beam","slug":"combinefn-setup-and-combinefn-teardown-are-not-supported","errorCode":null,"errorMessage":"CombineFn.setup and CombineFn.teardown are not supported with non-portable Dataflow runner. Please use Dataflow Portable Runner instead.","messagePattern":"CombineFn\\.setup and CombineFn\\.teardown are not supported with non-portable Dataflow runner\\. Please use Dataflow Portable Runner instead\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/runners/dataflow/dataflow_runner.py","lineNumber":330,"sourceCode":"\n    return FlattenInputVisitor()\n\n  @staticmethod\n  def combinefn_visitor():\n    # Imported here to avoid circular dependencies.\n    from apache_beam import core\n    from apache_beam.pipeline import PipelineVisitor\n\n    class CombineFnVisitor(PipelineVisitor):\n      \"\"\"Checks if `CombineFn` has non-default setup or teardown methods.\n      If yes, raises `ValueError`.\n      \"\"\"\n      def visit_transform(self, applied_transform):\n        transform = applied_transform.transform\n        if isinstance(transform, core.ParDo) and isinstance(\n            transform.fn, core.CombineValuesDoFn):\n          if self._overrides_setup_or_teardown(transform.fn.combinefn):\n            raise ValueError(\n                'CombineFn.setup and CombineFn.teardown are '\n                'not supported with non-portable Dataflow '\n                'runner. Please use Dataflow Portable Runner instead.')\n\n      @staticmethod\n      def _overrides_setup_or_teardown(combinefn):\n        # TODO(https://github.com/apache/beam/issues/18716): provide an\n        # implementation for this method\n        return False\n\n    return CombineFnVisitor()\n\n  def _adjust_pipeline_for_dataflow_v2(self, pipeline):\n    # Dataflow runner requires a KV type for GBK inputs, hence we enforce that\n    # here.\n    pipeline.visit(\n        group_by_key_input_visitor(\n            not pipeline._options.view_as(","sourceCodeStart":312,"sourceCodeEnd":348,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py#L312-L348","documentation":"The non-portable (legacy) Dataflow runner never invokes CombineFn.setup()/teardown(), so if a pipeline uses a CombineFn that overrides them the DataflowRunner raises this ValueError during translation instead of silently ignoring the lifecycle hooks. Dataflow Runner v2 (portable) supports them.","triggerScenarios":"Running a pipeline on the legacy Dataflow runner whose CombineFn overrides setup() or teardown(); visit_transform detects core.CombineValuesDoFn whose combinefn overrides those methods while runner v2 is disabled.","commonSituations":"Combining custom resource setup (DB connections, file handles) inside a CombineFn and submitting to legacy Dataflow; pipelines written for portable runners ported back to the legacy runner.","solutions":["Run on Dataflow Runner v2: remove any runner-v2-disabling experiments (do not set --dataflow_runner_v2=False or the disable experiment).","Move the setup/teardown logic into create_accumulator()/add_input()/extract_output() or into the DoFn's setup/teardown of a ParDo.","Upgrade apache-beam to a version where Runner v2 is the default for Dataflow.","Refactor the CombineFn so its state is self-contained and needs no per-bundle lifecycle hooks."],"exampleFix":"// before\noptions.view_as(TypeOptions).experiments = ['disable_runner_v2']\n// after\n# remove disable_runner_v2 experiment; Runner v2 supports CombineFn.setup/teardown","handlingStrategy":"validation","validationCode":"def combinefn_needs_runner_v2(fn):\n    import inspect\n    base = apache_beam.transforms.combinefn.CombineFn\n    return (type(fn).setup is not base.setup or type(fn).teardown is not base.teardown)","typeGuard":null,"tryCatchPattern":"try:\n    pipeline.run()\nexcept ValueError as e:\n    if 'CombineFn.setup' in str(e):\n        raise SystemExit('Remove disable_runner_v2 experiment or refactor CombineFn')","preventionTips":["Do not set the disable_runner_v2 experiment on modern Beam","Keep CombineFn stateless; do setup/teardown in DoFn setup for ParDo paths","Verify runner v2 status with _is_runner_v2_disabled(options) before submit"],"tags":["python","apache-beam","dataflow","combinefn","runner-v2"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}