{"record":{"id":"c97aa420e1f2bb33","repo":"apache/beam","slug":"groupby-agg-func-agg-func-r","errorCode":null,"errorMessage":"GroupBy.agg(func={agg_func!r})","messagePattern":"GroupBy\\.agg\\(func=(.+?)\\)","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/dataframe/frames.py","lineNumber":4770,"sourceCode":"  Raises:\n      NotImplementedError: If the aggregation function type is unsupported.\n  \"\"\"\n  if _is_associative(agg_func):\n    return _liftable_agg(agg_func)(gb, *args, **kwargs)\n  elif _is_liftable_with_sum(agg_func):\n    return _liftable_agg(agg_func, postagg_meth='sum')(gb, *args, **kwargs)\n  elif _is_unliftable(agg_func):\n    return _unliftable_agg(agg_func)(gb, *args, **kwargs)\n  elif callable(agg_func):\n    return DeferredDataFrame(\n        expressions.ComputedExpression(\n            agg_name,\n            lambda gb_val: gb_val.agg(agg_func, *args, **kwargs),\n            [gb._expr],\n            requires_partition_by=partitionings.Index(),\n            preserves_partition_by=partitionings.Singleton()))\n  else:\n    raise NotImplementedError(f\"GroupBy.agg(func={agg_func!r})\")\n\ndef _is_associative(agg_func):\n  return _check_str_or_np_builtin(agg_func, LIFTABLE_AGGREGATIONS)\n\ndef _is_liftable_with_sum(agg_func):\n  return _check_str_or_np_builtin(agg_func, LIFTABLE_WITH_SUM_AGGREGATIONS)\n\ndef _is_unliftable(agg_func):\n  return _check_str_or_np_builtin(agg_func, UNLIFTABLE_AGGREGATIONS)\n\nNUMERIC_AGGREGATIONS = ['max', 'min', 'prod', 'sum', 'mean', 'median', 'std',\n                        'var', 'sem', 'skew', 'kurt', 'kurtosis']\n# mad was removed in Pandas 2.0.\nif PD_VERSION < (2, 0):\n  NUMERIC_AGGREGATIONS.append('mad')\n\ndef _is_numeric(agg_func):\n  return _check_str_or_np_builtin(agg_func, NUMERIC_AGGREGATIONS)","sourceCodeStart":4752,"sourceCodeEnd":4788,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/dataframe/frames.py#L4752-L4788","documentation":"GroupBy.agg() supports only string/numpy-builtin aggregations (liftable ones), lists of them, and dict/list forms handled above; any other `func` type (arbitrary callable passed in an unsupported position, or exotic objects) falls through to this NotImplementedError naming the offending func.","triggerScenarios":"Calling gb.agg(custom_fn) or gb.agg(func=some_object) where the func is not a recognized string (e.g. 'sum', 'mean'), numpy builtin, or a supported list/dict of such on a Beam deferred groupby.","commonSituations":"Using arbitrary lambdas with agg in Beam (unlike pandas which allows them); pandas code migration where a named aggregation with a lambda was used; passing a class or partial object.","solutions":["Restrict agg funcs to supported strings/numpy builtins ('sum', 'mean', 'min', 'max', 'count', etc.) or lists/dicts of them.","For custom logic, use gb.apply(custom_fn) or gb.transform(custom_fn) with a callable instead.","Check `_check_str_or_np_builtin(agg_func, LIFTABLE_AGGREGATIONS)` semantics and upgrade the pipeline if Beam adds support."],"exampleFix":"# before\ndf.groupby('k').agg(lambda x: x.max() - x.min())\n# after\ndf.groupby('k').agg(ptp=('v', lambda x: x.max() - x.min())) if supported, else df.groupby('k').apply(lambda g: g['v'].max() - g['v'].min())","handlingStrategy":"validation","validationCode":"SUPPORTED = {'sum', 'mean', 'min', 'max', 'count', 'size', 'std', 'var'}\nif not (agg_func in SUPPORTED or (isinstance(agg_func, list) and all(a in SUPPORTED for a in agg_func))):\n    raise NotImplementedError('Unsupported agg func %r for Beam' % (agg_func,))","typeGuard":"def is_supported_agg(agg_func) -> bool:\n    from apache_beam.dataframe.frames import _check_str_or_np_builtin, LIFTABLE_AGGREGATIONS\n    return _check_str_or_np_builtin(agg_func, LIFTABLE_AGGREGATIONS)","tryCatchPattern":"try:\n    out = beam_df.groupby('k').agg(agg_func)\nexcept NotImplementedError as e:\n    if str(e).startswith('GroupBy.agg(func='):\n        out = beam_df.groupby('k').apply(lambda g, f=agg_func: f(g))","preventionTips":["Restrict agg funcs to Beam's liftable string/numpy aggregations.","Use groupby.apply for arbitrary custom per-group logic.","Consult apache_beam.dataframe.frames.LIFTABLE_AGGREGATIONS when choosing functions."],"tags":["apache-beam","dataframe","not-implemented","aggregation"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}