apache/beam · error · SideInputError

PCollection used directly as side input argument. Specify As

Error message

PCollection used directly as side input argument. Specify AsIter(pcollection) or AsSingleton(pcollection) to indicate how the PCollection is to be used.

What it means

Apache Beam's PTransform.__init__ rejects a raw PCollection passed as a positional or keyword argument to a transform's fn. Side inputs must be wrapped to declare how they should be materialized (iterable or singleton). The library cannot guess the access semantics, so it fails fast at pipeline-construction time with error.SideInputError.

Source

Thrown at sdks/python/apache_beam/transforms/ptransform.py:888

  :class:`PTransform` s like :func:`~apache_beam.transforms.core.FlatMap`
  invoke user-supplied code in some kind of package (e.g. a
  :class:`~apache_beam.transforms.core.DoFn`) and optionally provide arguments
  and side inputs to that code. This internal-use-only class contains common
  functionality for :class:`PTransform` s that fit this model.
  """
  def __init__(self, fn, *args, **kwargs):
    # type: (WithTypeHints, *Any, **Any) -> None
    if isinstance(fn, type) and issubclass(fn, WithTypeHints):
      # Don't treat Fn class objects as callables.
      raise ValueError('Use %s() not %s.' % (fn.__name__, fn.__name__))
    self.fn = self.make_fn(fn, bool(args or kwargs))
    # Now that we figure out the label, initialize the super-class.
    super().__init__()

    if (any(isinstance(v, pvalue.PCollection) for v in args) or
        any(isinstance(v, pvalue.PCollection) for v in kwargs.values())):
      raise error.SideInputError(
          'PCollection used directly as side input argument. Specify '
          'AsIter(pcollection) or AsSingleton(pcollection) to indicate how the '
          'PCollection is to be used.')
    self.args, self.kwargs, self.side_inputs = util.remove_objects_from_args(
        args, kwargs, pvalue.AsSideInput)
    self.raw_side_inputs = args, kwargs

    # Prevent name collisions with fns of the form '<function <lambda> at ...>'
    self._cached_fn = self.fn

    # Ensure fn and side inputs are picklable for remote execution.
    try:
      self.fn = pickler.roundtrip(self.fn)
    except (RuntimeError, TypeError, Exception) as e:
      raise RuntimeError(
          'Unable to pickle fn %s: %s. '
          'User code must be serializable (picklable) for distributed '
          'execution. This usually happens when lambdas or closures capture '

View on GitHub (pinned to 12126d8942)

Solutions

1. Wrap the PCollection in the intended view: beam.pvalue.AsIter(pcoll) or beam.pvalue.AsSingleton(pcoll).
2. If a dict lookup is intended, use beam.pvalue.AsDict(pcoll) or AsMap(pcoll).
3. If the PCollection was meant to be the main input, restructure so it is the first positional input of the transform rather than an extra argument.

Example fix

// before
beam.Map(add_offset, offsets_pcoll)
// after
beam.Map(add_offset, beam.pvalue.AsSingleton(offsets_pcoll))

def assert_no_raw_side_inputs(fn_args, fn_kwargs):
    for v in list(fn_args) + list(fn_kwargs.values()):
        if isinstance(v, apache_beam.pvalue.PCollection):
            raise ValueError(f'wrap {v} in AsIter/AsSingleton before passing as side input')

def is_side_input(v) -> bool:
    return isinstance(v, apache_beam.pvalue.AsSideInput)

try:
    pcoll | beam.Map(fn, other)
except apache_beam.error.SideInputError:
    pcoll | beam.Map(fn, beam.pvalue.AsSingleton(other))

When it happens

Trigger: Calling a DoFn/transform with a PCollection passed directly in args or kwargs, e.g. beam.Map(MyFn(), other_pcoll), instead of beam.Map(MyFn(), beam.pvalue.AsIter(other_pcoll)).

Common situations: Developers forget to wrap the second PCollection when joining two streams, or refactor code that previously used a plain value into one fed by another PCollection; also common when porting examples that use pvalue.AsSingleton but skipping the wrapper.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• A sink must inherit iobase.Sink, iobase.NativeSink, or be a
• Unknown access_pattern: %s
• Value for sideinput %s not provided
• Unsupported access pattern for %r: %r
• Cannot insert different protos %r and %r with the same ID %r

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0dd8f249292b9814. Report an issue: GitHub.