apache/beam · error · NotImplementedError

GroupBy.agg(func= )

Error message

GroupBy.agg(func={agg_func!r})

What it means

GroupBy.agg() supports only string/numpy-builtin aggregations (liftable ones), lists of them, and dict/list forms handled above; any other `func` type (arbitrary callable passed in an unsupported position, or exotic objects) falls through to this NotImplementedError naming the offending func.

Solutions

  1. Restrict agg funcs to supported strings/numpy builtins ('sum', 'mean', 'min', 'max', 'count', etc.) or lists/dicts of them.
  2. For custom logic, use gb.apply(custom_fn) or gb.transform(custom_fn) with a callable instead.
  3. Check `_check_str_or_np_builtin(agg_func, LIFTABLE_AGGREGATIONS)` semantics and upgrade the pipeline if Beam adds support.

Example fix

# before
df.groupby('k').agg(lambda x: x.max() - x.min())
# after
df.groupby('k').agg(ptp=('v', lambda x: x.max() - x.min())) if supported, else df.groupby('k').apply(lambda g: g['v'].max() - g['v'].min())
Defensive patterns

Strategy: validation

Validate before calling

SUPPORTED = {'sum', 'mean', 'min', 'max', 'count', 'size', 'std', 'var'}
if not (agg_func in SUPPORTED or (isinstance(agg_func, list) and all(a in SUPPORTED for a in agg_func))):
    raise NotImplementedError('Unsupported agg func %r for Beam' % (agg_func,))

Type guard

def is_supported_agg(agg_func) -> bool:
    from apache_beam.dataframe.frames import _check_str_or_np_builtin, LIFTABLE_AGGREGATIONS
    return _check_str_or_np_builtin(agg_func, LIFTABLE_AGGREGATIONS)

Try / catch

try:
    out = beam_df.groupby('k').agg(agg_func)
except NotImplementedError as e:
    if str(e).startswith('GroupBy.agg(func='):
        out = beam_df.groupby('k').apply(lambda g, f=agg_func: f(g))

Prevention

When it happens

Trigger: Calling gb.agg(custom_fn) or gb.agg(func=some_object) where the func is not a recognized string (e.g. 'sum', 'mean'), numpy builtin, or a supported list/dict of such on a Beam deferred groupby.

Common situations: Using arbitrary lambdas with agg in Beam (unlike pandas which allows them); pandas code migration where a named aggregation with a lambda was used; passing a class or partial object.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c97aa420e1f2bb33. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/dataframe/frames.py:4770

  Raises:
      NotImplementedError: If the aggregation function type is unsupported.
  """
  if _is_associative(agg_func):
    return _liftable_agg(agg_func)(gb, *args, **kwargs)
  elif _is_liftable_with_sum(agg_func):
    return _liftable_agg(agg_func, postagg_meth='sum')(gb, *args, **kwargs)
  elif _is_unliftable(agg_func):
    return _unliftable_agg(agg_func)(gb, *args, **kwargs)
  elif callable(agg_func):
    return DeferredDataFrame(
        expressions.ComputedExpression(
            agg_name,
            lambda gb_val: gb_val.agg(agg_func, *args, **kwargs),
            [gb._expr],
            requires_partition_by=partitionings.Index(),
            preserves_partition_by=partitionings.Singleton()))
  else:
    raise NotImplementedError(f"GroupBy.agg(func={agg_func!r})")

def _is_associative(agg_func):
  return _check_str_or_np_builtin(agg_func, LIFTABLE_AGGREGATIONS)

def _is_liftable_with_sum(agg_func):
  return _check_str_or_np_builtin(agg_func, LIFTABLE_WITH_SUM_AGGREGATIONS)

def _is_unliftable(agg_func):
  return _check_str_or_np_builtin(agg_func, UNLIFTABLE_AGGREGATIONS)

NUMERIC_AGGREGATIONS = ['max', 'min', 'prod', 'sum', 'mean', 'median', 'std',
                        'var', 'sem', 'skew', 'kurt', 'kurtosis']
# mad was removed in Pandas 2.0.
if PD_VERSION < (2, 0):
  NUMERIC_AGGREGATIONS.append('mad')

def _is_numeric(agg_func):
  return _check_str_or_np_builtin(agg_func, NUMERIC_AGGREGATIONS)

View on GitHub (pinned to 12126d8942)