apache/beam · error · WontImplementError

sort_index(axis=index) is not supported because it imposes…

Error message

sort_index(axis=index) is not supported because it imposes an ordering on the dataset which we cannot guarantee will be preserved.

What it means

sort_index(axis=index) is rejected because sorting a distributed dataset's rows imposes an ordering that Beam cannot guarantee will be preserved through the pipeline (see the Beam order-sensitive operations doc). Beam DataFrames deliberately refuse any operation whose result depends on row order. Only axis=columns is allowed for sort_index, since that reorders columns by name deterministically.

Solutions

Call sort_index(axis='columns') if you only need column ordering by name.
If row ordering matters, collect with to_pandas() and sort_index locally, or sort in the sink (e.g. WriteToText doesn't guarantee order either — use a single-partition step).
Redesign the pipeline so correctness does not depend on index order.

Example fix

// before
df.sort_index()  # WontImplementError
// after
df = df.to_pandas().sort_index()

Defensive patterns

Strategy: validation

Validate before calling

def check_sort_index(axis=0):
    if axis in (0, 'index'):
        raise ValueError("sort_index(axis=index) is order-sensitive and unsupported in Beam")
    return True

Type guard

def is_column_axis(axis):
    return axis in (1, 'columns')

Try / catch

from apache_beam.dataframe import frame_base
try:
    df = df.sort_index()
except frame_base.WontImplementError:
    df = df.to_pandas().sort_index()

Prevention

Design pipelines so correctness never depends on row/index order.
If ordering is needed for output, sort in the sink or after collection.
Only use axis='columns' with sort_index in deferred code.

When it happens

Trigger: Calling df.sort_index() (default axis=0), df.sort_index(axis=0), or df.sort_index(axis='index') on a DeferredDataFrame or DeferredSeries.

Common situations: Migrating pandas code that sorts by index before joins or output; assuming index order matters downstream; forgetting that Beam partitions data arbitrarily.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/802c8b77aa709c4d. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/dataframe/frames.py:877

    else:
      # axis=columns will reorder the columns based on the data
      raise frame_base.WontImplementError(
          "sort_values(axis=columns) is not supported because the order of the "
          "columns in the result depends on the data.",
          reason="non-deferred-columns")

  @frame_base.with_docs_from(pd.DataFrame)
  @frame_base.args_to_kwargs(pd.DataFrame)
  @frame_base.populate_defaults(pd.DataFrame)
  @frame_base.maybe_inplace
  def sort_index(self, axis, **kwargs):
    """``axis=index`` is not allowed because it imposes an ordering on the
    dataset, and we cannot guarantee it will be maintained (see
    https://s.apache.org/dataframe-order-sensitive-operations). Only
    ``axis=columns`` is allowed."""
    if axis in (0, 'index'):
      # axis=rows imposes an ordering on the DataFrame which we do not support
      raise frame_base.WontImplementError(
          "sort_index(axis=index) is not supported because it imposes an "
          "ordering on the dataset which we cannot guarantee will be "
          "preserved.",
          reason="order-sensitive")

    # axis=columns reorders the columns by name
    return frame_base.DeferredFrame.wrap(
        expressions.ComputedExpression(
            'sort_index',
            lambda df: df.sort_index(axis=axis, **kwargs),
            [self._expr],
            requires_partition_by=partitionings.Arbitrary(),
            preserves_partition_by=partitionings.Arbitrary(),
        ))

  @frame_base.with_docs_from(pd.DataFrame)
  @frame_base.args_to_kwargs(
      pd.DataFrame, removed_args=["errors"] if PD_VERSION >= (2, 0) else None)

View on GitHub (pinned to 12126d8942)