What causes "Should never be expanded directly." in apache/beam?

Expand the inner transform instead: use `wrapper.transform.expand(pvalue)`.

apache/beam · error · RuntimeError

Should never be expanded directly.

Error message

Should never be expanded directly.

What it means

`_NamedPTransform` (used to attach labels, e.g. `'label' >> transform`) is only a wrapper delegating to an inner transform; it has no expansion of its own. Calling expand() on it directly is a programming error, so it raises RuntimeError('Should never be expanded directly.').

Solutions

Expand the inner transform instead: use `wrapper.transform.expand(pvalue)`.
Apply the labeled transform through `|` / `__ror__` (e.g. `pcollection | 'name' >> MyTransform()`), never by calling expand manually.
In graph-traversal code, unwrap `_NamedPTransform` nodes via their `transform` attribute before expanding.

Example fix

# before
('label' >> beam.Map(fn)).expand(pc)
# after
pc | 'label' >> beam.Map(fn)
# or: ('label' >> beam.Map(fn)).transform.expand(pc)

Defensive patterns

Strategy: type-guard

Validate before calling

from apache_beam.transforms.ptransform import _NamedPTransform
if isinstance(t, _NamedPTransform):
  t = t.transform

Type guard

def unwrap_transform(t):
  from apache_beam.transforms.ptransform import _NamedPTransform
  return t.transform if isinstance(t, _NamedPTransform) else t

Try / catch

try:
  out = t.expand(pc)
except RuntimeError as e:
  if 'Should never be expanded directly' in str(e):
    t = unwrap_transform(t)
    out = t.expand(pc)
  else:
    raise

Prevention

Never call expand() directly in user code; apply transforms with `|`.
In traversal code, always unwrap _NamedPTransform wrappers via .transform.
Prefer the public pipeline API over internal Beam classes.

When it happens

Trigger: Calling `.expand(pvalue)` on a `_NamedPTransform` (i.e., a transform obtained from the `>>` label operator), or code that walks the transform graph and calls expand on wrapper nodes.

Common situations: Custom pipeline introspection/traversal code that mistakenly expands labeled wrappers; misuse of internal Beam APIs in testing or framework glue.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c433f9f5a6562645. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/transforms/ptransform.py:1187

    return fn.default_label()
  elif hasattr(fn, '__name__'):
    if fn.__name__ == '<lambda>':
      return '<lambda at %s:%s>' % (
          os.path.basename(fn.__code__.co_filename), fn.__code__.co_firstlineno)
    return fn.__name__
  return str(fn)


class _NamedPTransform(PTransform):
  def __init__(self, transform, label):
    super().__init__(label)
    self.transform = transform

  def __ror__(self, pvalueish, _unused=None):
    return self.transform.__ror__(pvalueish, self.label)

  def expand(self, pvalue):
    raise RuntimeError("Should never be expanded directly.")

  def annotations(self):
    return self.transform.annotations()

  def __rrshift__(self, label):
    return _NamedPTransform(self.transform, label)

  def with_resource_hints(self, **kwargs):
    self.transform.with_resource_hints(**kwargs)
    return self

  def __getattr__(self, attr):
    transform_attr = getattr(self.transform, attr)
    if callable(transform_attr):

      @wraps(transform_attr)
      def wrapper(*args, **kwargs):
        result = transform_attr(*args, **kwargs)

View on GitHub (pinned to 12126d8942)