apache/beam · error · NotImplementedError

by

Error message

by

What it means

Beam's groupby only accepts by as column names or index level names. Other pandas-legal forms of by (e.g. a Series, mapping, function, or axis=1 column grouping) are not supported and hit NotImplementedError(by), echoing the unsupported by value.

Source

Thrown at sdks/python/apache_beam/dataframe/frames.py:552

      if grouping_columns:
        # TODO(https://github.com/apache/beam/issues/20759):
        # It should be possible to do this without creating
        # an expression manually, by using DeferredDataFrame.set_index, i.e.:
        #   to_group_with_index = self.set_index([self.index] +
        #                                        grouping_columns)._expr
        to_group_with_index = expressions.ComputedExpression(
            'move_grouped_columns_to_index',
            lambda df: df.set_index([df.index] + grouping_columns, drop=False),
            [self._expr],
            requires_partition_by=partitionings.Arbitrary(),
            preserves_partition_by=partitionings.Index(
                list(range(self._expr.proxy().index.nlevels))))
      else:
        to_group_with_index = self._expr

    else:
      raise NotImplementedError(by)

    return DeferredGroupBy(
        expressions.ComputedExpression(
            'groupbyindex', lambda df: df.groupby(
                level=list(range(df.index.nlevels)), group_keys=group_keys, **
                kwargs), [to_group],
            requires_partition_by=partitionings.Index(),
            preserves_partition_by=partitionings.Arbitrary()),
        kwargs,
        to_group,
        to_group_with_index,
        grouping_columns=grouping_columns,
        grouping_indexes=grouping_indexes,
        group_keys=group_keys)

  @property  # type: ignore
  @frame_base.with_docs_from(pd.DataFrame)
  def loc(self):

View on GitHub (pinned to 12126d8942)

Solutions

1. Convert the callable/mapping to a column first (df['g'] = by_result) then group by that column name
2. If grouping by an external Series, merge/assign it as a column before groupby
3. Group only by index level names via level= instead of unsupported by forms

Example fix

// before
df.groupby(lambda x: x[0])
// after
df['first_letter'] = df.index.str[0]
df.groupby('first_letter')

if callable(by) or not isinstance(by, (str, list, tuple)) or (isinstance(by, (list, tuple)) and not all(isinstance(b, str) for b in by)):
    raise NotImplementedError("Beam groupby supports only column/index label names for by")

def by_is_labels(by):
    return isinstance(by, str) or (isinstance(by, (list, tuple)) and all(isinstance(b, str) for b in by))

try:
    out = df.groupby(by)
except NotImplementedError:
    df = df.assign(gkey=by_fn(df.index) if callable(by) else None)
    out = df.groupby('gkey')

When it happens

Trigger: df.groupby(lambda x: x % 2), df.groupby(some_series), df.groupby({'a': 'grp'}), or df.groupby('col', axis=1) — any by that is not label(s) present in columns/index names.

Common situations: Direct ports of pandas code using callables or external Series as group keys; attempts to group columns (axis=1); passing a Grouping-spec dict for renaming.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• groupby(as_index=False)
• Assigning an index is not yet supported. Consider using set_
• per-level align
• Indexing with a non-bool deferred frame is not yet supported
• set_index with Index or Series instances is not yet supporte

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6a5358461a53b4a0. Report an issue: GitHub.