apache/beam · error · WontImplementError

nsmallest(keep= ) is not supported because it is order…

Error message

nsmallest(keep={keep!r}) is not supported because it is order sensitive. Only keep="all" is supported.

What it means

DeferredDataFrame/Series.nsmallest() with keep='first' or 'last' picks tie winners by row order, which a distributed pipeline cannot guarantee. Only keep='all' is supported; any other keep value raises WontImplementError with reason 'order-sensitive'. (keep='any' is accepted and mapped to an arbitrary pick.)

Solutions

Call nsmallest(n, ..., keep='all')
Use keep='any' (Beam-specific) when any single duplicate is acceptable
Post-process with drop_duplicates()/groupby if you need to reduce tied rows
Restructure so ties don't matter (e.g. aggregate ties explicitly)

Example fix

// before
df.nsmallest(3, 'latency')
// after
df.nsmallest(3, 'latency', keep='all')

Defensive patterns

Strategy: validation

Validate before calling

assert keep == 'all', "nsmallest on Beam dataframes only supports keep='all' (or keep='any')"

Type guard

def beam_safe_keep(keep):
    return keep in ('all', 'any')

Try / catch

from apache_beam.dataframe import frame_base
try:
    bottom = df.nsmallest(n, col, keep='all')
except frame_base.WontImplementError:
    bottom = df.nsmallest(n, col, keep='all')

Prevention

Never rely on keep='first'/'last' tie-breaking in distributed pipelines
Pass keep='all' or Beam-specific keep='any' explicitly
Review ported pandas code for omitted keep arguments

When it happens

Trigger: Calling nsmallest(n, keep='first') or keep='last', or relying on pandas' default keep='first' by omitting keep; any keep value other than 'any' or 'all'.

Common situations: Direct pandas ports where keep defaults to 'first'; code depending on which duplicate row is retained; typos in the keep string.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/88f3d9d513a63df6. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/dataframe/frames.py:2232

      return frame_base.DeferredFrame.wrap(
          expressions.ComputedExpression(
              'nlargest', lambda df: df.nlargest(**kwargs), [per_partition],
              preserves_partition_by=partitionings.Arbitrary(),
              requires_partition_by=partitionings.Singleton()))

  @frame_base.with_docs_from(pd.Series)
  @frame_base.args_to_kwargs(pd.Series)
  @frame_base.populate_defaults(pd.Series)
  def nsmallest(self, keep, **kwargs):
    """Only ``keep=False`` and ``keep="any"`` are supported. Other values of
    ``keep`` make this an order-sensitive operation. Note ``keep="any"`` is
    a Beam-specific option that guarantees only one duplicate will be kept, but
    unlike ``"first"`` and ``"last"`` it makes no guarantees about _which_
    duplicate element is kept."""
    if keep == 'any':
      keep = 'first'
    elif keep != 'all':
      raise frame_base.WontImplementError(
          f"nsmallest(keep={keep!r}) is not supported because it is "
          "order sensitive. Only keep=\"all\" is supported.",
          reason="order-sensitive")
    kwargs['keep'] = keep
    per_partition = expressions.ComputedExpression(
        'nsmallest-per-partition', lambda df: df.nsmallest(**kwargs),
        [self._expr],
        preserves_partition_by=partitionings.Arbitrary(),
        requires_partition_by=partitionings.Arbitrary())
    with expressions.allow_non_parallel_operations(True):
      return frame_base.DeferredFrame.wrap(
          expressions.ComputedExpression(
              'nsmallest', lambda df: df.nsmallest(**kwargs), [per_partition],
              preserves_partition_by=partitionings.Arbitrary(),
              requires_partition_by=partitionings.Singleton()))

  @property  # type: ignore
  @frame_base.with_docs_from(pd.Series)

View on GitHub (pinned to 12126d8942)