apache/beam · error · TypeError

ParDo must be called with a DoFn instance.

Error message

ParDo must be called with a DoFn instance.

What it means

beam.ParDo (and its subclasses like FlatMap/Map wrappers) requires the fn argument to be an instance of DoFn. Passing a plain function or other object causes this TypeError because ParDo relies on DoFn process/ lifecycle methods.

Source

Thrown at sdks/python/apache_beam/transforms/core.py:1581

    *args: positional arguments passed to the :class:`DoFn` object.
    **kwargs:  keyword arguments passed to the :class:`DoFn` object.

  Note that the positional and keyword arguments will be processed in order
  to detect :class:`~apache_beam.pvalue.PCollection` s that will be computed as
  side inputs to the transform. During pipeline execution whenever the
  :class:`DoFn` object gets executed (its :meth:`DoFn.process()` method gets
  called) the :class:`~apache_beam.pvalue.PCollection` arguments will be
  replaced by values from the :class:`~apache_beam.pvalue.PCollection` in the
  exact positions where they appear in the argument lists.
  """
  def __init__(self, fn, *args, **kwargs):
    super().__init__(fn, *args, **kwargs)
    # TODO(robertwb): Change all uses of the dofn attribute to use fn instead.
    self.dofn = self.fn
    self.output_tags = set()  # type: typing.Set[str]

    if not isinstance(self.fn, DoFn):
      raise TypeError('ParDo must be called with a DoFn instance.')

    # DoFn.process cannot allow both return and yield
    if _check_fn_use_yield_and_return(self.fn.process):
      _LOGGER.warning(
          'Using yield and return in the process method '
          'of %s can lead to unexpected behavior, see:'
          'https://github.com/apache/beam/issues/22969.',
          self.fn.__class__)

    # Validate the DoFn by creating a DoFnSignature
    from apache_beam.runners.common import DoFnSignature
    self._signature = DoFnSignature(self.fn)

  def with_exception_handling(
      self,
      main_tag='good',
      dead_letter_tag='bad',
      *,

View on GitHub (pinned to 12126d8942)

Solutions

1. Subclass DoFn and pass an instance: class MyDoFn(beam.DoFn): def process(self, x): yield x; then beam.ParDo(MyDoFn()).
2. If you only have a callable, use beam.FlatMap / beam.Map (they wrap callables via _CallableWrapperDoFn) instead of ParDo directly.
3. Check that the variable is not None or a function; isinstance(fn, beam.DoFn) guard.
4. If migrating code, convert the function into a DoFn with @beam.DoFn.process or keep using Map/FlatMap.
5. Example fix: `p | beam.ParDo(MyDoFn())` with `class MyDoFn(beam.DoFn)` instead of `p | beam.ParDo(lambda x: x)`.

Example fix

// before
pcoll | beam.ParDo(lambda x: [x, x])
// after
class DuplicateDoFn(beam.DoFn):
    def process(self, x):
        yield x
        yield x
pcoll | beam.ParDo(DuplicateDoFn())

if not isinstance(fn, beam.DoFn):
    raise TypeError('ParDo requires a DoFn instance; use beam.Map/FlatMap for plain callables')

def is_dofn(x) -> bool:
    return isinstance(x, beam.DoFn)

When it happens

Trigger: ParDo(my_plain_function) called directly instead of ParDo(DoFn subclass instance); passing a lambda or builtin to ParDo; constructing ParDo in a custom runner code path.

Common situations: Users porting from Map/FlatMap (which wrap callables) to ParDo without subclassing DoFn; refactoring where a DoFn instance was replaced with a bare function; custom transforms that misuse ParDo internally.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Returning a %s from a ParDo or FlatMap is not allowed. Pleas
• Transform '{full_label}' expects a PCollection as input. Got
• 'obj_to_invoke' has to be either a 'DoFn' or a 'RestrictionP
• DoFn.process() method-only parameter %s cannot be used in %s
• Main output tag %r must be different from side output tags %

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f5e626111e7c135a. Report an issue: GitHub.