apache/beam · error · NotImplementedError

per-level align

Error message

per-level align

What it means

align() with a level= argument would require aligning per index level, which Beam's partitioned implementation does not support; it raises NotImplementedError('per-level align'). method= for fill is separately rejected as order-sensitive.

Source

Thrown at sdks/python/apache_beam/dataframe/frames.py:1416

            'append', lambda s, to_append: s.append(
                to_append, verify_integrity=verify_integrity, **kwargs),
            [self._expr, to_append._expr],
            requires_partition_by=requires,
            preserves_partition_by=partitionings.Arbitrary()))

  @frame_base.with_docs_from(pd.Series)
  @frame_base.args_to_kwargs(pd.Series)
  @frame_base.populate_defaults(pd.Series)
  def align(self, other, join, axis, level, method, **kwargs):
    """Aligning per-level is not yet supported. Only the default,
    ``level=None``, is allowed.

    Filling NaN values via ``method`` is not supported, because it is
    `order-sensitive
    <https://s.apache.org/dataframe-order-sensitive-operations>`_.
    Only the default, ``method=None``, is allowed."""
    if level is not None:
      raise NotImplementedError('per-level align')
    if method is not None and method != lib.no_default:
      raise frame_base.WontImplementError(
          f"align(method={method!r}) is not supported because it is "
          "order sensitive. Only align(method=None) is supported.",
          reason="order-sensitive")
    # We're using pd.concat here as expressions don't yet support
    # multiple return values.
    aligned = frame_base.DeferredFrame.wrap(
        expressions.ComputedExpression(
            'align', lambda x, y: pd.concat([x, y], axis=1, join='inner'),
            [self._expr, other._expr],
            requires_partition_by=partitionings.Index(),
            preserves_partition_by=partitionings.Arbitrary()))
    return aligned.iloc[:, 0], aligned.iloc[:, 1]

  argsort = frame_base.wont_implement_method(
      pd.Series, 'argsort', reason="order-sensitive")

View on GitHub (pinned to 12126d8942)

Solutions

1. Drop level= and align on fully matching indexes, or pre-reset the indexes so they are flat
2. Use join/merge (which Beam supports) instead of per-level alignment
3. Reindex explicitly with supported operations if a join-like alignment is needed

Example fix

// before
l, r = df1.align(df2, level=0)
// after
l, r = df1.align(df2.reset_index(level=0, drop=True))

if level is not None:
    raise NotImplementedError("per-level align unsupported; flatten indexes first")
if method is not None and method is not lib.no_default:
    raise NotImplementedError("align(method=...) unsupported; use method=None")

try:
    l, r = a.align(b, level=lv)
except NotImplementedError:
    l, r = a.align(b.reset_index(level=lv, drop=True))

When it happens

Trigger: df1.align(df2, level=0) or align(..., level='name') — any non-None level argument; method='ffill'/'bfill' hits the adjacent WontImplementError instead.

Common situations: pandas code aligning MultiIndexed frames on a level; forward-filling alignment patterns ported to Beam; join-like workflows better expressed with merge/join in Beam.

Related errors

• groupby(as_index=False)
• by
• Assigning an index is not yet supported. Consider using set_
• Indexing with a non-bool deferred frame is not yet supported
• set_index with Index or Series instances is not yet supporte

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f3be1bfd917ba971. Report an issue: GitHub.