apache/beam · error · WontImplementError

sort_values(axis=columns) is not supported because the…

Error message

sort_values(axis=columns) is not supported because the order of the columns in the result depends on the data.

What it means

apache_beam.dataframe raises this WontImplementError when sort_values is called with axis=columns. Reordering columns based on cell data cannot be done in Beam's deferred distributed execution, because the resulting column order would depend on the data flowing through the pipeline rather than being fixed at graph-construction time. Only axis=index (or the default 0) would be possible, and even that is separately rejected as order-sensitive.

Solutions

Avoid sorting columns in the pipeline: perform sort_values(axis=columns) on the final collected pandas DataFrame after to_pandas().
Use sort_values(axis='index') is also unsupported; if you only need sorted row output for sinks, write to a sink that guarantees ordering or collect first.
Restructure the transform to not depend on column order (e.g. operate on columns by name).

Example fix

// before
df.sort_values(axis='columns')  # WontImplementError in Beam
// after
pdf = df.to_pandas()
pdf = pdf.sort_values(axis='columns')

Defensive patterns

Strategy: validation

Validate before calling

import inspect
if kwargs.get('axis', 0) in (1, 'columns'):
    raise ValueError("sort_values(axis=columns) is unsupported in Beam; sort after to_pandas()")

Type guard

def can_sort_values(df, axis=0):
    return axis in (0, 'index')

Try / catch

from apache_beam.dataframe import frame_base
try:
    df = df.sort_values(axis='columns')
except frame_base.WontImplementError:
    df = df.to_pandas().sort_values(axis='columns')

Prevention

Never pass axis=1/'columns' to sort_values in Beam pipelines.
Reserve column-ordering transformations for the final eager pandas stage.
Operate on columns by name, not by data-derived position.

When it happens

Trigger: Calling df.sort_values(axis=1) or df.sort_values(axis='columns') on a DeferredDataFrame, or a deferred frame where with_docs_from/args_to_kwargs maps a positional axis argument to 1.

Common situations: Porting an existing pandas script to Beam DataFrames unchanged; sorting columns by their values (e.g. ranking columns per row) in a pipeline; confusion between Beam's two axis restrictions on sort_values.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/55ee77eeaa0434f4. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/dataframe/frames.py:861

    """``sort_values`` is not implemented.

    It is not implemented for ``axis=index`` because it imposes an ordering on
    the dataset, and it likely will not be maintained (see
    https://s.apache.org/dataframe-order-sensitive-operations).

    It is not implemented for ``axis=columns`` because it makes the order of
    the columns depend on the data (see
    https://s.apache.org/dataframe-non-deferred-columns)."""
    if axis in (0, 'index'):
      # axis=index imposes an ordering on the DataFrame rows which we do not
      # support
      raise frame_base.WontImplementError(
          "sort_values(axis=index) is not supported because it imposes an "
          "ordering on the dataset which likely will not be preserved.",
          reason="order-sensitive")
    else:
      # axis=columns will reorder the columns based on the data
      raise frame_base.WontImplementError(
          "sort_values(axis=columns) is not supported because the order of the "
          "columns in the result depends on the data.",
          reason="non-deferred-columns")

  @frame_base.with_docs_from(pd.DataFrame)
  @frame_base.args_to_kwargs(pd.DataFrame)
  @frame_base.populate_defaults(pd.DataFrame)
  @frame_base.maybe_inplace
  def sort_index(self, axis, **kwargs):
    """``axis=index`` is not allowed because it imposes an ordering on the
    dataset, and we cannot guarantee it will be maintained (see
    https://s.apache.org/dataframe-order-sensitive-operations). Only
    ``axis=columns`` is allowed."""
    if axis in (0, 'index'):
      # axis=rows imposes an ordering on the DataFrame which we do not support
      raise frame_base.WontImplementError(
          "sort_index(axis=index) is not supported because it imposes an "
          "ordering on the dataset which we cannot guarantee will be "

View on GitHub (pinned to 12126d8942)