apache/beam · error · ValueError

Disabling Dataflow Portable Runner no longer supported using

Error message

Disabling Dataflow Portable Runner no longer supported using Beam Python %s.

What it means

This ValueError guards against opting out of Dataflow Runner v2. Newer Apache Beam Python SDK versions dropped support for the legacy (non-portable) Dataflow runner, so any pipeline option or experiment that disables Runner v2 causes run_pipeline to refuse to submit the job, reporting the SDK version in use.

Source

Thrown at sdks/python/apache_beam/runners/dataflow/dataflow_runner.py:354

      def _overrides_setup_or_teardown(combinefn):
        # TODO(https://github.com/apache/beam/issues/18716): provide an
        # implementation for this method
        return False

    return CombineFnVisitor()

  def _adjust_pipeline_for_dataflow_v2(self, pipeline):
    # Dataflow runner requires a KV type for GBK inputs, hence we enforce that
    # here.
    pipeline.visit(
        group_by_key_input_visitor(
            not pipeline._options.view_as(
                TypeOptions).allow_non_deterministic_key_coders))

  def run_pipeline(self, pipeline, options, pipeline_proto=None):
    """Remotely executes entire pipeline or parts reachable from node."""
    if _is_runner_v2_disabled(options):
      raise ValueError(
          'Disabling Dataflow Portable Runner no longer supported '
          'using Beam Python %s.' % beam.version.__version__)

    # Label goog-dataflow-notebook if job is started from notebook.
    if is_in_notebook():
      notebook_version = (
          'goog-dataflow-notebook=' +
          beam.version.__version__.replace('.', '_'))
      if options.view_as(GoogleCloudOptions).labels:
        options.view_as(GoogleCloudOptions).labels.append(notebook_version)
      else:
        options.view_as(GoogleCloudOptions).labels = [notebook_version]

    # Import here to avoid adding the dependency for local running scenarios.
    try:
      # pylint: disable=wrong-import-order, wrong-import-position
      from apache_beam.runners.dataflow.internal import apiclient
    except ImportError:

View on GitHub (pinned to 12126d8942)

Solutions

1. Remove the runner-v2-disabling option/experiment from pipeline options.
2. Upgrade or downgrade apache-beam so the SDK version matches the runner mode you need (older SDKs still allow legacy runner).
3. Migrate the pipeline to Runner v2 and verify supported features (e.g. CombineFn.setup/teardown).
4. Test with the DirectRunner first to catch behavior differences after dropping the legacy flag.

Example fix

// before
--experiments=disable_runner_v2
// after
# remove the experiment; use the default Dataflow Runner v2

if 'disable_runner_v2' in (opts.view_as(TypeOptions).experiments or []):
    raise SystemExit('disable_runner_v2 no longer supported by this Beam SDK')

try:
    pipeline.run()
except ValueError as e:
    if 'Portable Runner no longer supported' in str(e):
        opts.view_as(TypeOptions).experiments = [x for x in (opts.view_as(TypeOptions).experiments or []) if x != 'disable_runner_v2']
        pipeline.run()

When it happens

Trigger: Submitting a Dataflow pipeline with an option/experiment that disables Runner v2 (e.g. --experiments=disable_runner_v2 / dataflow_runner_v2=False) while using a Beam Python version where that toggle was removed; detected via _is_runner_v2_disabled(options).

Common situations: Upgrading apache-beam while keeping old job-submission scripts that pinned the legacy runner; copying pipeline options from older documentation or templates.

Understand the failure class

Background: "is deprecated and will be removed" — deprecation warnings for old API names, keywords, and options, and how to migrate before the removal release — this error's family across 29 libraries.

Related errors

• CombineFn.setup and CombineFn.teardown are not supported wit
• Could not find element
• Too many matches
• Could not translate the internal step name %r since job grap
• Could not translate the internal step name %r.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f032447d720e1114. Report an issue: GitHub.