apache/beam · error · ValueError

Unknown or inapplicable phase for pre_optimize

Error message

Unknown or inapplicable phase for pre_optimize: %s

What it means

pre_optimize accepts only a comma-separated whitelist of optimization phases: pack_combiners, lift_combiners, expand_sdf. Any other name passed via --pre_optimize raises this ValueError in _optimize_pipeline.

Solutions

1. Use only pack_combiners, lift_combiners, or expand_sdf in --pre_optimize.
2. Remove the misspelled/unsupported phase from the comma-separated list.
3. Check translations module of your Beam version for the supported phase set.

Example fix

// before
--pre_optimize=pack_combiners,sort_stages
// after
--pre_optimize=pack_combiners,lift_combiners

ALLOWED = {'pack_combiners', 'lift_combiners', 'expand_sdf'}
requested = set(pre_optimize.split(','))
assert requested <= ALLOWED, f'unsupported pre_optimize phases: {requested - ALLOWED}'

When it happens

Trigger: Setting --pre_optimize to a name not in ('pack_combiners','lift_combiners','expand_sdf') — e.g. a typo like 'lift_combiner', an unknown phase, or sort_stages which is added automatically.

Common situations: Typos in pipeline options; copying phase names from newer/older Beam versions where the whitelist differs; trying to enable internal phases like sort_stages explicitly.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Unknown runner
• A 'datagen' table requires either 'rows-per-second' (for…
• A schema was provided without a data format (or viceversa)…
• Batch size is too large! It should be smaller or equal than
• Batch size must be a positive integer

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/84dc2cf9d7ba1ba7. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/portability/portable_runner.py:339

        ]
        partial = True
      elif pre_optimize == 'all':
        phases = translations.standard_optimize_phases()
        partial = False
      elif pre_optimize == 'all_except_fusion':
        # TODO(https://github.com/apache/beam/issues/19422): Delete this branch
        # after PortableRunner supports beam:runner:executable_stage:v1.
        phases = translations.standard_optimize_phases()
        phases.remove(translations.greedily_fuse)
        partial = True
      else:
        phases = []
        for phase_name in pre_optimize.split(','):
          # For now, these are all we allow.
          if phase_name in ('pack_combiners', 'lift_combiners', 'expand_sdf'):
            phases.append(getattr(translations, phase_name))
          else:
            raise ValueError(
                'Unknown or inapplicable phase for pre_optimize: %s' %
                phase_name)
        phases.append(translations.sort_stages)
        partial = True

      # All (known) portable runners (ie Flink and Spark) support these URNs.
      known_urns = frozenset([
          common_urns.composites.RESHUFFLE.urn,
          common_urns.primitives.IMPULSE.urn,
          common_urns.primitives.FLATTEN.urn,
          common_urns.primitives.GROUP_BY_KEY.urn
      ])
      proto_pipeline = translations.optimize_pipeline(
          proto_pipeline,
          phases=phases,
          known_runner_urns=known_urns,
          partial=partial)

View on GitHub (pinned to 12126d8942)