pandas-dev/pandas · error · ValueError

cannot diff on axis=

Error message

cannot diff {type(arr).__name__} on axis={axis}

What it means

When diff() is called on an ExtensionArray (a non-numpy pandas array type like CategoricalArray or IntervalArray), the code checks whether the array supports the subtraction operator. If it does, it verifies that the requested axis is valid for the array's dimensionality. Since ExtensionArrays are 1-D, any axis >= 1 is invalid. This ValueError indicates an attempt to diff along a non-existent axis on a 1-D extension array.

Solutions

  1. If calling diff on a DataFrame, ensure the axis matches: df.diff(axis=0) operates along rows, df.diff(axis=1) along columns.
  2. Convert the ExtensionArray to numpy first: df['col'].astype(np.float64).diff().
  3. If this is an internal/indirect trigger, check whether your ExtensionArray's __sub__ or __xor__ method is interfering with the axis check.

Example fix

# before — axis=1 on a single extension-array column
 df['interval_col']._values.diff(n=1, axis=1)

# after — convert to numeric or use DataFrame-level diff
df[['interval_col']].diff(axis=0)
Defensive patterns

Strategy: validation

Validate before calling

def safe_diff_extension_array(arr, n=1, axis=0):
    if axis >= arr.ndim:
        raise ValueError(f"axis={axis} is invalid for ndim={arr.ndim}")
    return arr.diff(n, axis=axis) if hasattr(arr, 'diff') else None

Type guard

def is_valid_diff_axis(arr, axis) -> bool:
    return axis < np.asarray(arr).ndim

Try / catch

try:
    result = df.diff(n=1, axis=axis)
except ValueError as e:
    if "cannot diff" in str(e) and "axis" in str(e):
        result = df.diff(n=1, axis=0)  # fall back to default axis
    else:
        raise

Prevention

When it happens

Trigger: Calling diff(axis=1) on a 1-D ExtensionArray (e.g., a Categorical or Interval column extracted as ._values). Internally passing axis >= arr.ndim when arr is a 1-D extension array. This is more commonly an internal routing error than a direct user mistake.

Common situations: DataFrame.diff(axis=1) on a column whose dtype is an ExtensionArray type — the internal code may route the per-column diff call with the wrong axis. Using a third-party ExtensionArray subclass that interacts with diff in unexpected ways.

Related errors

AI-assisted analysis of pandas-dev/pandas@3b7651241d (2026-08-11). Data as JSON: /api/errors/48b332abcc28cf19. Report an issue: GitHub.

Appendix: source

Thrown at pandas/core/algorithms.py:1541

    na = np.nan
    dtype = arr.dtype

    is_bool = is_bool_dtype(dtype)
    if is_bool:
        op = operator.xor
    else:
        op = operator.sub

    if isinstance(dtype, NumpyEADtype):
        # NumpyExtensionArray cannot necessarily hold shifted versions of itself.
        arr = arr.to_numpy()
        dtype = arr.dtype

    if not isinstance(arr, np.ndarray):
        # i.e ExtensionArray
        if hasattr(arr, f"__{op.__name__}__"):
            if axis >= arr.ndim:
                raise ValueError(f"cannot diff {type(arr).__name__} on axis={axis}")
            return op(arr, arr.shift(n))
        else:
            raise TypeError(
                f"{type(arr).__name__} has no 'diff' method. "
                "Convert to a suitable dtype prior to calling 'diff'."
            )

    is_timedelta = False
    if arr.dtype.kind in "mM":
        dtype = np.int64
        arr = arr.view("i8")
        na = iNaT
        is_timedelta = True

    elif is_bool:
        # We have to cast in order to be able to hold np.nan
        dtype = np.object_

View on GitHub (pinned to 3b7651241d)