apache/beam · error · WontImplementError

Accessing a DeferredSeries with an iterator is sensitive to…

Error message

Accessing a DeferredSeries with an iterator is sensitive to the order of the data.

What it means

Indexing a DeferredSeries with an iterator key or a boolean indexer (a list/array of booleans) is rejected because a boolean mask's elements are paired with rows positionally, and positions are not meaningful in Beam's arbitrarily partitioned data. Beam raises WontImplementError with reason "order-sensitive".

Solutions

Filter with a boolean DeferredSeries instead: s[s > 0] or s[mask_series] where mask_series is itself deferred.
Use supported filtering APIs (e.g. s.loc with a deferred boolean expression).
If the mask is small/static, apply the filter after to_pandas().

Example fix

// before
mask = [True, False, True]
s[mask]  # WontImplementError
// after
s = s[s > 0]  # deferred boolean expression

Defensive patterns

Strategy: validation

Validate before calling

import pandas as pd
if pd.core.series.is_iterator(key) or pd.core.common.is_bool_indexer(key):
    raise ValueError("pass a deferred boolean Series instead of a list/iterator mask")

Type guard

def is_deferred_mask(series, key):
    return isinstance(key, DeferredSeries)

Try / catch

from apache_beam.dataframe import frame_base
try:
    out = s[bool_list]
except frame_base.WontImplementError:
    out = s[s > threshold]  # build the mask inside the deferred graph

Prevention

Never index deferred frames with Python lists/iterators of booleans.
Express masks as DeferredSeries expressions (s > x, s.isin(...)).
Keep mask and data in the same (deferred) world.

When it happens

Trigger: s[iter([True, False, ...])] or s[[True, False, True]] (list/ndarray of booleans accepted by pd.core.common.is_bool_indexer) on a DeferredSeries.

Common situations: Porting pandas mask-filtering code; building a boolean list from another column and using it directly as an indexer.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2bb3b2545a2037cc. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/dataframe/frames.py:1348

              # yapf: disable
              'getitem',
              lambda df: df[key],
              [self._expr],
              requires_partition_by=partitionings.Arbitrary(),
              preserves_partition_by=partitionings.Arbitrary()))

    elif isinstance(key, DeferredSeries) and key._expr.proxy().dtype == bool:
      return frame_base.DeferredFrame.wrap(
          expressions.ComputedExpression(
              # yapf: disable
              'getitem',
              lambda df, indexer: df[indexer],
              [self._expr, key._expr],
              requires_partition_by=partitionings.Index(),
              preserves_partition_by=partitionings.Arbitrary()))

    elif pd.core.series.is_iterator(key) or pd.core.common.is_bool_indexer(key):
      raise frame_base.WontImplementError(
          "Accessing a DeferredSeries with an iterator is sensitive to the "
          "order of the data.",
          reason="order-sensitive")

    else:
      # We could consider returning a deferred scalar, but that might
      # be more surprising than a clear error.
      raise frame_base.WontImplementError(
          f"Indexing a series with key of type {type(key)} is not supported "
          "because it produces a non-deferred result.",
          reason="non-deferred-result")

  @frame_base.with_docs_from(pd.Series)
  def keys(self):
    return self.index

  # Series.T == transpose. Both are a no-op
  T = frame_base._elementwise_method('T', base=pd.Series)

View on GitHub (pinned to 12126d8942)