apache/beam · error · ValueError

DoFn has unsupported process_batch method parameter

Error message

DoFn {self.do_fn!r} has unsupported process_batch method parameter {d}

What it means

The runner's DoFn signature introspection found a process_batch (or process_for_key_batch) parameter that batch-mode execution cannot supply yet (beyond the explicitly handled Element/Key/State/Timer cases); this NotImplementedError marks a not-yet-supported parameter as the SDK evolves.

Solutions

  1. Restrict process_batch parameters to elements plus Window/Timestamp/PaneInfo params
  2. Move unsupported special-parameter usage into a regular process() DoFn
  3. Check the exact param object types against apache_beam.transforms.core.DoFn definitions

Example fix

// before
def process_batch(self, els, fin=DoFn.BundleFinalizerParam):
// after
def process_batch(self, els, w=DoFn.WindowParam):
Defensive patterns

Strategy: validation

Validate before calling

ALLOWED = (DoFn.WindowParam, DoFn.TimestampParam, DoFn.PaneInfoParam)
bad = [p for p in inspect.signature(MyDoFn.process_batch).values if p.default not in ALLOWED and isinstance(p.default, DoFn.DoFnParam)]
assert not bad

Prevention

When it happens

Trigger: process_batch declaring any core.DoFn.* param outside {WindowParam, TimestampParam, PaneInfoParam} (and the explicitly rejected Element/Key/State/Timer ones), validated via _validate during invoker setup.

Common situations: Adding BundleFinalizerParam or a custom param to a batch method; typos producing a param object the whitelist doesn't recognize.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/502da8c1430d575e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/common.py:374

      # Helpful errors for params which will be supported in the future
      if d == (core.DoFn.ElementParam):
        # We currently assume we can just get the typehint from the first
        # parameter. ElementParam breaks this assumption
        raise NotImplementedError(
            f"DoFn {self.do_fn!r} uses unsupported DoFn param ElementParam.")

      if d in (core.DoFn.KeyParam, core.DoFn.StateParam, core.DoFn.TimerParam):
        raise NotImplementedError(
            f"DoFn {self.do_fn!r} has unsupported per-key DoFn param {d}. "
            "Per-key DoFn params are not yet supported for process_batch "
            "(https://github.com/apache/beam/issues/21653).")

      # Fallback to catch anything not explicitly supported
      if not d in (core.DoFn.WindowParam,
                   core.DoFn.TimestampParam,
                   core.DoFn.PaneInfoParam):
        raise ValueError(
            f"DoFn {self.do_fn!r} has unsupported process_batch "
            f"method parameter {d}")

  def _validate_bundle_method(self, method_wrapper):
    """Validate that none of the DoFnParameters are used in the function
    """
    for param in core.DoFn.DoFnProcessParams:
      if param in method_wrapper.defaults:
        raise ValueError(
            'DoFn.process() method-only parameter %s cannot be used in %s.' %
            (param, method_wrapper))

  def _validate_stateful_dofn(self):
    # type: () -> None
    userstate.validate_stateful_dofn(self.do_fn)

  def is_splittable_dofn(self):
    # type: () -> bool

View on GitHub (pinned to 12126d8942)