apache/beam · error · KeyError

label

Error message

label

What it means

In the deferred groupby implementation, when grouping 'by' a concrete NumPy ndarray, this WontImplementError fires: the message/label field identifies the unsupported operation because grouping by a raw ndarray is order-sensitive, which the Beam DataFrame API cannot guarantee under its partitioning semantics. Use a named column or list of columns instead.

Source

Thrown at sdks/python/apache_beam/dataframe/frames.py:522

      grouping_columns = []
      grouping_indexes = [0]

    elif isinstance(by, np.ndarray):
      raise frame_base.WontImplementError(
          "Grouping by a concrete ndarray is order sensitive.",
          reason="order-sensitive")

    elif isinstance(self, DeferredDataFrame):
      if not isinstance(by, list):
        by = [by]
      # Find the columns that we need to move into the index so we can group by
      # them
      column_names = self._expr.proxy().columns
      grouping_columns = list(set(by).intersection(column_names))
      index_names = self._expr.proxy().index.names
      for label in by:
        if label not in index_names and label not in self._expr.proxy().columns:
          raise KeyError(label)
      grouping_indexes = list(set(by).intersection(index_names))

      if grouping_indexes:
        if set(by) == set(index_names):
          to_group = self._expr
        elif set(by).issubset(index_names):
          to_group = self.droplevel(index_names.difference(by))._expr
        else:
          to_group = self.reset_index(grouping_indexes).set_index(by)._expr
      else:
        to_group = self.set_index(by)._expr

      if grouping_columns:
        # TODO(https://github.com/apache/beam/issues/20759):
        # It should be possible to do this without creating
        # an expression manually, by using DeferredDataFrame.set_index, i.e.:
        #   to_group_with_index = self.set_index([self.index] +
        #                                        grouping_columns)._expr

View on GitHub (pinned to 12126d8942)

Solutions

1. Fix the label spelling to match an actual column or index name
2. Verify available keys with df.columns and df.index.names before grouping
3. Select only the columns that exist: by=[c for c in wanted if c in df.columns]

Example fix

// before
df.groupby('Counrty')
// after
assert 'country' in df.columns
df.groupby('country')

cols = set(df.columns) | set(df.index.names)
missing = [b for b in by if b not in cols]
if missing:
    raise KeyError(f"groupby keys not found: {missing}; have {sorted(cols)}")

def keys_exist(df, by):
    names = set(df.columns) | set(df.index.names)
    return all(b in names for b in (by if isinstance(by, (list, tuple)) else [by]))

try:
    out = df.groupby(by)
except KeyError as e:
    logging.error("unknown groupby key %s; columns=%s index=%s", e, list(df.columns), list(df.index.names))
    raise

When it happens

Trigger: df.groupby('countrys') (typo) where the column is named 'country'; by list containing a label renamed upstream; grouping by a label that only exists after a previous transformation that was dropped.

Common situations: Schema drift after upstream pipeline changes; case-sensitivity mistakes ('Date' vs 'date'); grouping by a MultiIndex level name that was dropped by an earlier reset_index/droplevel.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• k_val
• Cannot specify both 'labels' and 'index'/'columns'
• axis must be one of (0, 1, 'index', 'columns'), got '%s'
• groupby(as_index=False)
• You have to supply one of 'by' and 'level'

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/dcd79e729f0f0cde. Report an issue: GitHub.