apache/beam · error · NotImplementedError

type(row_index)

Error message

type(row_index)

What it means

When the .loc key is callable, Beam invokes it during pipeline construction with an empty proxy to compute the index; if the callable returns a list of booleans, NotImplementedError(type(row_index)) is raised because boolean-list row selection is unsupported.

Solutions

1. Make the callable return a pandas Series of booleans indexed like the frame, not a list.
2. Compute the mask with deferred expressions: df.loc[lambda df: df['v'] > 0].
3. Validate the callable's return type at construction time against (slice, pd.Series) before passing it.

Example fix

# before
df.loc[lambda df: [v > 0 for v in df['v']]]
# after
df.loc[lambda df: df['v'] > 0]

result = key_fn(empty_proxy_df)
assert isinstance(result, (slice, pd.Series)), 'callable .loc key must return slice or Series, got %r' % type(result)

def returns_valid_loc_index(fn, proxy_df) -> bool:
    import pandas as pd
    out = fn(proxy_df)
    return isinstance(out, (slice, pd.Series)) and not (isinstance(out, list))

try:
    out = beam_df.loc[key_fn]
except NotImplementedError as e:
    if str(e) == str(type(row_index)):
        out = beam_df.loc[lambda df: pd.Series(key_fn(df), index=df.index)]

When it happens

Trigger: df.loc[lambda df: [bool(x) for x in ...]] — a callable key returning a Python bool list on a Beam deferred DataFrame.

Common situations: Callable-based pandas idiom migrated to Beam where the lambda builds a list instead of a Series; computing masks in plain Python rather than with dataframe ops.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Assigning an index is not yet supported. Consider using…
• by
• concat(ignore_index)
• concat(levels)
• corrwith( )

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7c9965ab3a410c00. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/dataframe/frames.py:4954

        reindexed_expr = expressions.ComputedExpression(
            'data_to_index',
            data_to_index,
            [key._expr],
            requires_partition_by=partitionings.Arbitrary(),
            preserves_partition_by=partitionings.Singleton(),
        )
        args = [self._frame._expr, reindexed_expr]
    elif callable(key):

      def checked_callable_key(df):
        computed_index = key(df)
        if isinstance(computed_index, tuple):
          row_index, _ = computed_index
        else:
          row_index = computed_index
        if isinstance(row_index, list) and row_index and isinstance(
            row_index[0], bool):
          raise NotImplementedError(type(row_index))
        elif not isinstance(row_index, (slice, pd.Series)):
          raise NotImplementedError(type(row_index))
        return computed_index

      args = [self._frame._expr]
      func = lambda df: df.loc[checked_callable_key]
    else:
      raise NotImplementedError(type(key))

    return frame_base.DeferredFrame.wrap(
        expressions.ComputedExpression(
            'loc',
            func,
            args,
            requires_partition_by=(
                partitionings.JoinIndex()
                if len(args) > 1
                else partitionings.Arbitrary()),

View on GitHub (pinned to 12126d8942)