pandas-dev/pandas · error · NotImplementedError

Default 'empty' implementation is invalid for dtype=

Error message

Default 'empty' implementation is invalid for dtype='{dtype}'

What it means

_empty is the internal counterpart of ExtensionDtype.empty used to build an all-NA array of a given shape. The base default first builds an empty sequence via cls._from_sequence([], dtype=dtype), then takes indices broadcast to -1 (the all-NA placeholder) with allow_fill=True. If the resulting object is not an instance of cls or its dtype does not match, the default is deemed invalid and NotImplementedError is raised. This catches subclasses whose take/_from_sequence do not honor the NA contract.

Solutions

Override _empty on the subclass to construct the all-NA array directly without relying on the take-based default.
Ensure _from_sequence([], dtype=...) returns an instance of cls with exactly that dtype, and take([-1,...], allow_fill=True, fill_value=na_value) returns cls with the same dtype.
Verify self.dtype.na_value round-trips through take; align na_value in the ExtensionDtype.

Example fix

// before
# pandas internal reindex -> NotImplementedError: Default 'empty' implementation is invalid...

// after
@classmethod
def _empty(cls, shape, dtype):
    import numpy as np
    na = dtype.na_value
    data = np.broadcast_to(na, shape)
    return cls._from_sequence(data, dtype=dtype)

Defensive patterns

Strategy: validation

Validate before calling

# Verify _from_sequence/take round-trip the dtype before pandas calls _empty
import numpy as np
empty = type(arr)._from_sequence([], dtype=arr.dtype)
taker = np.intp(-1)
result = empty.take(np.array([taker]), allow_fill=True)
assert isinstance(result, type(arr)) and result.dtype == arr.dtype

Try / catch

try:
    out = dtype.empty((5,))
except NotImplementedError:
    # subclass does not honor the default; build NA-filled manually
    import numpy as np
    out = type(arr)._from_sequence(np.full(5, dtype.na_value), dtype=dtype)

Prevention

Ensure _from_sequence([], dtype=...) returns an instance of cls with exactly that dtype.
Ensure take(allow_fill=True) preserves class and dtype.
Override _empty if the take-based default does not fit your storage layout.

When it happens

Trigger: pandas internally requests an empty/NA-filled EA of a given shape (during reindex, alignment, or construction of a placeholder block) for a custom dtype whose _from_sequence([])/take does not reproduce the same dtype or return the same class. The validation in the base default then fails.

Common situations: A custom EA whose _from_sequence changes the dtype (e.g. upcasts), or whose take(allow_fill=True) returns a different wrapper class. A dtype whose na_value handling in take is inconsistent. Triggered indirectly through reindex/merge on frames containing the custom column.

Related errors

AI-assisted analysis of pandas-dev/pandas@3b7651241d (2026-08-11). Data as JSON: /api/errors/2acfc0ae6e77c508. Report an issue: GitHub.

Appendix: source

Thrown at pandas/core/arrays/base.py:2862

    @classmethod
    def _empty(cls, shape: Shape, dtype: ExtensionDtype):
        """
        Create an ExtensionArray with the given shape and dtype.

        See also
        --------
        ExtensionDtype.empty
            ExtensionDtype.empty is the 'official' public version of this API.
        """
        # Implementer note: while ExtensionDtype.empty is the public way to
        # call this method, it is still required to implement this `_empty`
        # method as well (it is called internally in pandas)
        obj = cls._from_sequence([], dtype=dtype)

        taker = np.broadcast_to(np.intp(-1), shape)
        result = obj.take(taker, allow_fill=True)
        if not isinstance(result, cls) or dtype != result.dtype:
            raise NotImplementedError(
                f"Default 'empty' implementation is invalid for dtype='{dtype}'"
            )
        return result

    def _quantile(self, qs: npt.NDArray[np.float64], interpolation: str) -> Self:
        """
        Compute the quantiles of self for each quantile in `qs`.

        Parameters
        ----------
        qs : np.ndarray[float64]
        interpolation: str

        Returns
        -------
        same type as self
        """
        mask = np.asarray(self.isna())

View on GitHub (pinned to 3b7651241d)