apache/beam · error · ValueError

CombineFn.setup and CombineFn.teardown are not supported…

Error message

CombineFn.setup and CombineFn.teardown are not supported with non-portable Dataflow runner. Please use Dataflow Portable Runner instead.

What it means

The non-portable (legacy) Dataflow runner never invokes CombineFn.setup()/teardown(), so if a pipeline uses a CombineFn that overrides them the DataflowRunner raises this ValueError during translation instead of silently ignoring the lifecycle hooks. Dataflow Runner v2 (portable) supports them.

Solutions

1. Run on Dataflow Runner v2: remove any runner-v2-disabling experiments (do not set --dataflow_runner_v2=False or the disable experiment).
2. Move the setup/teardown logic into create_accumulator()/add_input()/extract_output() or into the DoFn's setup/teardown of a ParDo.
3. Upgrade apache-beam to a version where Runner v2 is the default for Dataflow.
4. Refactor the CombineFn so its state is self-contained and needs no per-bundle lifecycle hooks.

Example fix

// before
options.view_as(TypeOptions).experiments = ['disable_runner_v2']
// after
# remove disable_runner_v2 experiment; Runner v2 supports CombineFn.setup/teardown

def combinefn_needs_runner_v2(fn):
    import inspect
    base = apache_beam.transforms.combinefn.CombineFn
    return (type(fn).setup is not base.setup or type(fn).teardown is not base.teardown)

try:
    pipeline.run()
except ValueError as e:
    if 'CombineFn.setup' in str(e):
        raise SystemExit('Remove disable_runner_v2 experiment or refactor CombineFn')

When it happens

Trigger: Running a pipeline on the legacy Dataflow runner whose CombineFn overrides setup() or teardown(); visit_transform detects core.CombineValuesDoFn whose combinefn overrides those methods while runner v2 is disabled.

Common situations: Combining custom resource setup (DB connections, file handles) inside a CombineFn and submitting to legacy Dataflow; pipelines written for portable runners ported back to the legacy runner.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Disabling Dataflow Portable Runner no longer supported…
• Can not query metrics. Job id is unknown.
• Coder for the GroupByKey operation
• combine_fn must be specified.
• Could not find element

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e6c22f9b534a1568. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/dataflow/dataflow_runner.py:330

    return FlattenInputVisitor()

  @staticmethod
  def combinefn_visitor():
    # Imported here to avoid circular dependencies.
    from apache_beam import core
    from apache_beam.pipeline import PipelineVisitor

    class CombineFnVisitor(PipelineVisitor):
      """Checks if `CombineFn` has non-default setup or teardown methods.
      If yes, raises `ValueError`.
      """
      def visit_transform(self, applied_transform):
        transform = applied_transform.transform
        if isinstance(transform, core.ParDo) and isinstance(
            transform.fn, core.CombineValuesDoFn):
          if self._overrides_setup_or_teardown(transform.fn.combinefn):
            raise ValueError(
                'CombineFn.setup and CombineFn.teardown are '
                'not supported with non-portable Dataflow '
                'runner. Please use Dataflow Portable Runner instead.')

      @staticmethod
      def _overrides_setup_or_teardown(combinefn):
        # TODO(https://github.com/apache/beam/issues/18716): provide an
        # implementation for this method
        return False

    return CombineFnVisitor()

  def _adjust_pipeline_for_dataflow_v2(self, pipeline):
    # Dataflow runner requires a KV type for GBK inputs, hence we enforce that
    # here.
    pipeline.visit(
        group_by_key_input_visitor(
            not pipeline._options.view_as(

View on GitHub (pinned to 12126d8942)