apache/beam · error · NotImplementedError

Indexing with a non-bool deferred frame is not yet supported

Error message

Indexing with a non-bool deferred frame is not yet supported. Consider using df.loc[...]

What it means

For DeferredDataFrame.__getitem__, boolean-mask indexing with a DeferredSeries is supported (delegated to .loc), but indexing with any other DeferredBase (e.g. a non-bool deferred frame or deferred column expression) is not implemented. Such keys interact surprisingly with column selection logic, so the API fails early with a clear NotImplementedError instead of producing wrong results.

Source

Thrown at sdks/python/apache_beam/dataframe/frames.py:2530

  def keys(self):
    return self.columns

  def __getattr__(self, name):
    # Column attribute access.
    if name in self._expr.proxy().columns:
      return self[name]
    else:
      return object.__getattribute__(self, name)

  def __getitem__(self, key):
    # TODO: Replicate pd.DataFrame.__getitem__ logic
    if isinstance(key, DeferredSeries) and key._expr.proxy().dtype == bool:
      return self.loc[key]

    elif isinstance(key, frame_base.DeferredBase):
      # Fail early if key is a DeferredBase as it interacts surprisingly with
      # key in self._expr.proxy().columns
      raise NotImplementedError(
          "Indexing with a non-bool deferred frame is not yet supported. "
          "Consider using df.loc[...]")

    elif isinstance(key, slice):
      if _is_null_slice(key):
        return self
      elif _is_integer_slice(key):
        # This depends on the contents of the index.
        raise frame_base.WontImplementError(
            "Integer slices are not supported as they are ambiguous. Please "
            "use iloc or loc with integer slices.")
      else:
        return self.loc[key]

    elif (
        (isinstance(key, list) and all(key_column in self._expr.proxy().columns
                                       for key_column in key)) or
        key in self._expr.proxy().columns):

View on GitHub (pinned to 12126d8942)

Solutions

Use df.loc[mask] explicitly instead of df[mask] for deferred boolean masks.
Ensure the mask is a DeferredSeries of dtype bool (e.g. df['col'] > 0), not a whole DeferredDataFrame.
Convert to plain pandas (to_pandas()) for complex non-boolean deferred indexing.
Reduce the mask to a single boolean column first, then apply it via .loc.

Example fix

// before
filtered = df[df > 0]  # deferred frame key
// after
filtered = df.loc[df['value'] > 0]

Defensive patterns

Strategy: type-guard

Validate before calling

if isinstance(key, DeferredBase) and not (isinstance(key, DeferredSeries) and key._expr.proxy().dtype == bool):
    raise TypeError('use df.loc[...] with a bool DeferredSeries')

Type guard

def is_valid_bool_mask(key) -> bool:
    return isinstance(key, DeferredSeries) and key._expr.proxy().dtype == bool

Try / catch

try:
    filtered = df[mask]
except NotImplementedError:
    filtered = df.loc[mask]

Prevention

Always use .loc for deferred boolean filtering.
Ensure masks are single-column bool DeferredSeries (df['col'] > 0).
Avoid whole-frame comparisons like df[df > 0] in Beam DataFrames.
Fall back to pandas for exotic deferred indexing patterns.

When it happens

Trigger: df[mask_df] where mask_df is a DeferredDataFrame whose proxy dtype is not bool; indexing with a deferred column/other DeferredFrame object; forgetting .loc and passing a deferred key directly.

Common situations: Porting pandas patterns like df[df > 0] where df is a DeferredDataFrame (the comparison yields a deferred frame, not a bool deferred series); boolean filtering with multi-column masks.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/97d01b8a4c5ce7ef. Report an issue: GitHub.