pandas-dev/pandas · error · ValueError

Column {colname} is backed by an extension array, which is n

Error message

Column {colname} is backed by an extension array, which is not supported by the numba engine.

What it means

Raised by `validate_values_for_numba` when a column is backed by a pandas extension array (e.g. `Int64`, `Float64` nullable, `Categorical`, string dtype) even if its logical type is numeric. The numba engine requires plain numpy-backed arrays; extension arrays have a different memory layout and a mask, which numba cannot consume directly.

Source

Thrown at pandas/core/apply.py:984

    def generate_numba_apply_func(
        func, nogil: bool = True, parallel: bool = False
    ) -> Callable[[npt.NDArray, Index, Index], dict[int, Any]]:
        pass

    @abc.abstractmethod
    def apply_with_numba(self):
        pass

    def validate_values_for_numba(self) -> None:
        # Validate column dtypes all OK
        for colname, dtype in self.obj.dtypes.items():
            if not is_numeric_dtype(dtype):
                raise ValueError(
                    f"Column {colname} must have a numeric dtype. "
                    f"Found '{dtype}' instead"
                )
            if is_extension_array_dtype(dtype):
                raise ValueError(
                    f"Column {colname} is backed by an extension array, "
                    f"which is not supported by the numba engine."
                )

    @abc.abstractmethod
    def wrap_results_for_axis(
        self, results: ResType, res_index: Index
    ) -> DataFrame | Series:
        pass

    # ---------------------------------------------------------------

    @property
    def res_columns(self) -> Index:
        return self.result_columns

    @property
    def columns(self) -> Index:

View on GitHub (pinned to 71959b8cb9)

Solutions

Convert extension columns to plain numpy dtypes: `df = df.convert_dtypes(dtype_backend='numpy_nullable').astype({c: 'float64' for c in ext_cols})`, or `df[col].to_numpy(dtype='float64')`.
Drop or separate extension-array columns and use the python engine for them.
Use `astype('float64')` (with NaN handling for nullable ints) before invoking numba.

Example fix

# before
df.astype('Int64').apply(func, engine='numba', raw=True)
# after
df.astype('Int64').astype('float64').apply(func, engine='numba', raw=True)

Defensive patterns

Strategy: validation

Validate before calling

import pandas as pd

def apply_numba_numpy_only(df, func, **kw):
    bad = [c for c in df.columns if pd.api.types.is_extension_array_dtype(df[c].dtype)]
    if bad:
        raise ValueError(f'extension-array columns unsupported by numba: {bad}')
    return df.apply(func, engine='numba', raw=True, **kw)

Type guard

def no_extension_arrays(df) -> bool:
    import pandas as pd
    return not any(pd.api.types.is_extension_array_dtype(dt) for dt in df.dtypes)

Try / catch

try:
    df.apply(func, engine='numba', raw=True)
except ValueError as e:
    if 'extension array' in str(e):
        df.astype('float64').apply(func, engine='numba', raw=True)
    else:
        raise

Prevention

Convert nullable/extension dtypes to plain numpy dtypes before invoking numba.
Audit dtypes with `df.dtypes` when planning a numba apply.

When it happens

Trigger: `df.apply(func, engine='numba', raw=True)` where `df` uses nullable pandas dtypes (`'Int64'`, `'Float64'`), `pd.Categorical`, or the new string dtype. `is_extension_array_dtype` returns True and the check fires.

Common situations: Migrating to nullable dtypes for missing-value semantics then attempting numba acceleration; reading data via APIs that default to extension dtypes; mixing extension and numpy columns.

Related errors

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/612fe6aa6af0d5c1. Report an issue: GitHub.