apache/beam · error · WontImplementError

axis must be 'index' when upper and/or lower are a…

Error message

axis must be 'index' when upper and/or lower are a DeferredFrame

What it means

DeferredSeries/DataFrame.clip supports DeferredFrame bounds (lower/upper as deferred series) only when axis='index' (axis 0), because aligning bound arrays along columns would be order-sensitive. Passing axis=1/'columns' with a DeferredFrame bound raises WontImplementError.

Solutions

  1. Use axis=0 or axis='index' when bounds are deferred frames.
  2. Convert the bounds to a plain pandas Series/constant if they are small enough to materialize on the driver.
  3. Rewrite as an elementwise expression: ddf.clip(lower=..., upper=...) per column via assign/transform.

Example fix

// before
ddf.clip(lower=deferred_bounds, axis=1)

// after
ddf.clip(lower=deferred_bounds, axis=0)  # or materialize bounds: clip(lower=bounds_pd_series)
Defensive patterns

Strategy: type-guard

Validate before calling

from apache_beam.dataframe.frame_base import DeferredFrame
if any(isinstance(kwargs.get(a), DeferredFrame) for a in ('lower', 'upper')) and axis not in (0, 'index'):
    axis = 'index'

Type guard

def is_deferred_bound(v) -> bool:
    from apache_beam.dataframe.frame_base import DeferredFrame
    return isinstance(v, DeferredFrame)

Try / catch

from apache_beam.dataframe import frame_base
try:
    out = ddf.clip(lower=lb, upper=ub, axis=axis)
except frame_base.WontImplementError:
    out = ddf.clip(lower=lb, upper=ub, axis='index')

Prevention

When it happens

Trigger: Calling `ddf.clip(lower=some_deferred_series, axis=1)` or `axis='columns'` where lower and/or upper is a DeferredFrame/DeferredSeries.

Common situations: Clipping each row/column against a series of thresholds computed in the same pipeline; porting `df.clip(lower=bounds, axis=1)` from pandas to Beam.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1f979c8876210780. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/dataframe/frames.py:3012

  memory_usage = frame_base.wont_implement_method(
      pd.DataFrame, 'memory_usage', reason="non-deferred-result")
  info = frame_base.wont_implement_method(
      pd.DataFrame, 'info', reason="non-deferred-result")


  @frame_base.with_docs_from(pd.DataFrame)
  @frame_base.args_to_kwargs(pd.DataFrame)
  @frame_base.populate_defaults(pd.DataFrame)
  @frame_base.maybe_inplace
  def clip(self, axis, **kwargs):
    """``lower`` and ``upper`` must be :class:`DeferredSeries` instances, or
    constants.  Array-like arguments are not supported because they are
    order-sensitive."""

    if any(isinstance(kwargs.get(arg, None), frame_base.DeferredFrame)
           for arg in ('upper', 'lower')) and axis not in (0, 'index'):
      raise frame_base.WontImplementError(
          "axis must be 'index' when upper and/or lower are a DeferredFrame",
          reason='order-sensitive')

    return frame_base._elementwise_method('clip', base=pd.DataFrame)(self,
                                                                     axis=axis,
                                                                     **kwargs)

  @frame_base.with_docs_from(pd.DataFrame)
  @frame_base.args_to_kwargs(pd.DataFrame)
  @frame_base.populate_defaults(pd.DataFrame)
  def corr(self, method, min_periods):
    """Only ``method="pearson"`` can be parallelized. Other methods require
    collecting all data on a single worker (see
    https://s.apache.org/dataframe-non-parallel-operations for details).
    """
    if method == 'pearson':
      proxy = self._expr.proxy().corr()
      columns = list(proxy.columns)

View on GitHub (pinned to 12126d8942)