What causes "func must be specified and it must be callable" in apache/beam?

Pass a callable predicate: df.groupby('k').filter(lambda df: df['v'].sum() > 0).

apache/beam · error · TypeError

func must be specified and it must be callable

Error message

func must be specified and it must be callable

What it means

DeferredGroupBy.filter() raises TypeError when `func` is None or not callable. Beam requires an explicit callable predicate deciding which groups to keep; there is no default behavior and string shortcuts are not accepted.

Solutions

Pass a callable predicate: df.groupby('k').filter(lambda df: df['v'].sum() > 0).
If you intended to aggregate, use .agg(...) or .apply(...) instead of .filter(...).
Guard the call site with `if callable(f): gb.filter(f)`.

Example fix

# before
df.groupby('k').filter('sum')
# after
df.groupby('k').filter(lambda df: df['v'].sum() > 0)

Defensive patterns

Strategy: type-guard

Validate before calling

if func is None or not callable(func):
    raise TypeError('filter requires a callable predicate')

Type guard

def is_valid_filter_fn(func) -> bool:
    return func is not None and callable(func)

Try / catch

try:
    out = beam_df.groupby('k').filter(func)
except TypeError as e:
    if 'func must be specified' in str(e):
        raise ValueError('Provide a callable predicate to groupby.filter') from e

Prevention

Never call groupby.filter without an argument.
Keep filter predicates as explicit lambdas/functions.
Lint for groupby(...).filter() calls with empty argument lists.

When it happens

Trigger: Calling df.groupby('k').filter() with no argument, or filter('name')/filter(non_callable) on a Beam deferred DataFrame.

Common situations: Omitting the predicate accidentally after refactoring; copying pandas code that passed a string; assuming filter() with no args is a no-op like dropna-only filtering.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8ac87c87cf61b631. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/dataframe/frames.py:4457

              **kwargs).droplevel(self._grouping_columns),
            [self._ungrouped_with_index],
            proxy=proxy,
            requires_partition_by=partitionings.Index(levels),
            preserves_partition_by=partitionings.Index(self._grouping_indexes)))

  @frame_base.with_docs_from(DataFrameGroupBy)
  def pipe(self, func, *args, **kwargs):
    if isinstance(func, tuple):
      func, data = func
      kwargs[data] = self
      return func(*args, **kwargs)

    return func(self, *args, **kwargs)

  @frame_base.with_docs_from(DataFrameGroupBy)
  def filter(self, func=None, dropna=True):
    if func is None or not callable(func):
      raise TypeError("func must be specified and it must be callable")

    def apply_fn(df):
      if func(df):
        return df
      elif not dropna:
        result = df.copy()
        result.iloc[:, :] = np.nan
        return result
      else:
        return df.iloc[:0]

    return self.apply(apply_fn).droplevel(self._grouping_columns)

  @property  # type: ignore
  @frame_base.with_docs_from(DataFrameGroupBy)
  def dtypes(self):
    return frame_base.DeferredFrame.wrap(
        expressions.ComputedExpression(

View on GitHub (pinned to 12126d8942)