apache/beam · error · TypeError

You have to supply one of 'by' and 'level'

Error message

You have to supply one of 'by' and 'level'

What it means

The Beam DataFrame API's groupby requires a grouping specification: pandas groupby needs either 'by' or 'level', and this guard in frames.py rejects calls where neither is supplied. It mirrors pandas' own error but is raised in the deferred (lazy) API before any expression is computed.

Source

Thrown at sdks/python/apache_beam/dataframe/frames.py:391

    Aggregations grouping by a categorical column with ``observed=False`` set
    are not currently parallelizable
    (`Issue 21827 <https://github.com/apache/beam/issues/21827>`_).
    """
    if not as_index:
      raise NotImplementedError('groupby(as_index=False)')

    if axis in (1, 'columns'):
      return _DeferredGroupByCols(
          expressions.ComputedExpression(
              'groupbycols', lambda df: df.groupby(
                  by, axis=axis, group_keys=group_keys, **kwargs), [self._expr],
              requires_partition_by=partitionings.Arbitrary(),
              preserves_partition_by=partitionings.Arbitrary()),
          group_keys=group_keys)

    if level is None and by is None:
      raise TypeError("You have to supply one of 'by' and 'level'")

    elif level is not None:
      if isinstance(level, (list, tuple)):
        grouping_indexes = level
      else:
        grouping_indexes = [level]

      grouping_columns = []

      index = self._expr.proxy().index

      # Translate to level numbers only
      grouping_indexes = [
          l if isinstance(l, int) else index.names.index(l)
          for l in grouping_indexes
      ]

      if index.nlevels == 1:

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass by= with one or more column/index labels
2. Or pass level= (an int, name, or list) to group by index level(s)
3. Check that the variable feeding by= is not None before calling

Example fix

// before
df.groupby(by=key)
# where key may be None
// after
if key is not None:
    df.groupby(by=key)
else:
    df.groupby(level=0)

if by is None and level is None:
    raise ValueError("groupby requires 'by' or 'level'")
out = df.groupby(by=by, level=level)

def has_group_key(by, level):
    return by is not None or level is not None

try:
    out = df.groupby(by=key)
except TypeError as e:
    if "'by' and 'level'" in str(e):
        out = df.groupby(level=0)

When it happens

Trigger: df.groupby() with no arguments, or calling groupby(None, level=None) — e.g. a wrapper that conditionally builds by= but passes None when the condition fails.

Common situations: Programmatic construction of groupby kwargs where the key variable was never set; refactors removing by= while relying on a default level; copy-paste from code that grouped via level but dropped that argument.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• Pipeline keyword required for non-deferred dataframe convers
• Cannot specify both 'labels' and 'index'/'columns'
• axis must be one of (0, 1, 'index', 'columns'), got '%s'
• groupby(as_index=False)
• label

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6efe693db6d9b3a4. Report an issue: GitHub.