apache/beam · error · WontImplementError

Grouping by a concrete ndarray is order sensitive.

Error message

Grouping by a concrete ndarray is order sensitive.

What it means

groupby(by=<numpy.ndarray>) is rejected because grouping by a concrete ndarray keys groups by data order, which Beam cannot preserve across distributed workers. The Beam DataFrame API only supports grouping by column labels, list of labels, Series, or Index — not raw arrays.

Source

Thrown at sdks/python/apache_beam/dataframe/frames.py:508

      to_group = expressions.ComputedExpression(
          'set_index',
          set_index, [self._expr, by._expr],
          requires_partition_by=partitionings.Index(),
          preserves_partition_by=partitionings.Singleton())

      orig_nlevels = self._expr.proxy().index.nlevels
      to_group_with_index = expressions.ComputedExpression(
          'prependindex',
          prepend_index, [self._expr, by._expr],
          requires_partition_by=partitionings.Index(),
          preserves_partition_by=partitionings.Index(
              list(range(1, orig_nlevels + 1))))

      grouping_columns = []
      grouping_indexes = [0]

    elif isinstance(by, np.ndarray):
      raise frame_base.WontImplementError(
          "Grouping by a concrete ndarray is order sensitive.",
          reason="order-sensitive")

    elif isinstance(self, DeferredDataFrame):
      if not isinstance(by, list):
        by = [by]
      # Find the columns that we need to move into the index so we can group by
      # them
      column_names = self._expr.proxy().columns
      grouping_columns = list(set(by).intersection(column_names))
      index_names = self._expr.proxy().index.names
      for label in by:
        if label not in index_names and label not in self._expr.proxy().columns:
          raise KeyError(label)
      grouping_indexes = list(set(by).intersection(index_names))

      if grouping_indexes:
        if set(by) == set(index_names):

View on GitHub (pinned to 12126d8942)

Solutions

1. Assign the array as a column first: df['key'] = array (as a deferred column), then df.groupby('key').
2. Group by an existing column name or list of column names.
3. Wrap the array in a pd.Series/Index that aligns by index if the data supports it.
4. Collect to pandas with to_pandas() if the operation must be non-deferred.

Example fix

// before
groups = df.groupby(np.array(['a','b','a']))
// after
df = df.assign(key=['a','b','a'])
groups = df.groupby('key')

if isinstance(by, np.ndarray):
    raise ValueError('Assign the array as a column and group by that column instead')

def groupby_is_supported(by) -> bool:
    return not isinstance(by, np.ndarray)

try:
    grouped = df.groupby(by)
except frame_base.WontImplementError:
    df = df.assign(key=by)
    grouped = df.groupby('key')

When it happens

Trigger: df.groupby(np.array([...])) or df.groupby(by=some_ndarray) on a DeferredDataFrame; also reachable via callers like aggregate, duplicated and drop_duplicates that pass ndarray keys through to groupby.

Common situations: Converting pandas code that groups by an external array of keys; building grouping keys programmatically as arrays; duplicated()/drop_duplicates() calls on arrays.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• fillna(method={method!r}, axis={axis!r}) is not supported be
• fillna(limit={method!r}, axis={axis!r}) is not supported bec
• replace(method={method!r}) is not supported because it is or
• sort_values(axis=index) is not supported because it imposes
• align(method={method!r}) is not supported because it is orde

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d1ee7c525b9f5d0e. Report an issue: GitHub.