pandas-dev/pandas · error · ValueError

too many dims to broadcast

Error message

too many dims to broadcast

What it means

Raised inside `apply_broadcast` when the per-column result of the UDF has `ndim > 1` (e.g. a 2-D array or DataFrame). Broadcast mode requires each column's result to be a scalar or a 1-D sequence matching the column length; a higher-dimensional object cannot be placed into the single column slot pandas is filling.

Solutions

1. Make the UDF return a scalar or 1-D array per column; if it produces multiple columns, do not use `result_type='broadcast'`.
2. If you need a 2-D output, drop `result_type='broadcast'` and assemble the result frame yourself.
3. Inspect `np.asarray(func(df[col])).ndim` for each column to find which column violates the contract.

Example fix

// before
df.apply(lambda s: np.outer(s, s), result_type='broadcast')
// after
out = df.apply(lambda s: np.outer(s, s))  # returns list of 2-D arrays; assemble manually

def broadcast_apply_safe(df, func):
    import numpy as np
    for col in df.columns:
        probe = np.asarray(func(df[col].head(2)))
        if probe.ndim > 1:
            raise ValueError(f'{func.__name__} returns {probe.ndim}D for column {col!r}; broadcast requires scalar/1D')
    return df.apply(func, result_type='broadcast')

def func_returns_1d_or_scalar(func, series) -> bool:
    import numpy as np
    try:
        return np.asarray(func(series.head(2))).ndim <= 1
    except Exception:
        return False

try:
    out = df.apply(func, result_type='broadcast')
except ValueError as e:
    if 'too many dims' in str(e):
        out = df.apply(func)  # let pandas assemble the 2D result itself
    else:
        raise

When it happens

Trigger: `df.apply(func, result_type='broadcast')` where `func(series)` returns a 2-D numpy array, a DataFrame, or any object whose `np.asarray(...).ndim > 1`.

Common situations: UDFs that return multi-column outputs (e.g. `np.outer`, polynomial feature expansions, pairwise computations); UDFs that accidentally stack/expand a 1-D result.

Related errors

• cannot broadcast result
• by_row= not allowed
• Column is backed by an extension array, which is not…
• Column must have a numeric dtype. Found ' ' instead
• Function did not transform

AI-assisted analysis of pandas-dev/pandas@3b7651241d (2026-08-11). Data as JSON: /api/errors/12d9b33960fa1f8b. Report an issue: GitHub.

Appendix: source

Thrown at pandas/core/apply.py:1265

            return self.obj._constructor(result, index=self.index, columns=self.columns)
        else:
            return self.obj._constructor_sliced(result, index=self.agg_axis)

    def apply_broadcast(self, target: DataFrame) -> DataFrame:
        assert callable(self.func)

        result_values = np.empty_like(target.values)

        # axis which we want to compare compliance
        result_compare = target.shape[0]

        for i, col in enumerate(target.columns):
            res = self.func(target[col], *self.args, **self.kwargs)
            ares = np.asarray(res).ndim

            # must be a scalar or 1d
            if ares > 1:
                raise ValueError("too many dims to broadcast")
            if ares == 1:
                # must match return dim
                if result_compare != len(res):
                    raise ValueError("cannot broadcast result")

            result_values[:, i] = res

        # we *always* preserve the original index / columns
        result = self.obj._constructor(
            result_values, index=target.index, columns=target.columns
        )
        return result

    def apply_standard(self):
        if self.engine == "python":
            results, res_index = self.apply_series_generator()
        else:
            results, res_index = self.apply_series_numba()

View on GitHub (pinned to 3b7651241d)