pandas-dev/pandas · error · ValueError

fill value in the sparse values not supported

Error message

fill value in the sparse values not supported

What it means

ValueError from SparseArray mapping (used by Series.map/.apply on sparse-backed Series) when a mapped sparse value would equal the array's fill_value. Allowing it would corrupt the sparse/dense distinction because the fill_value is implicit and not stored in sp_values.

Source

Thrown at pandas/core/arrays/sparse/array.py:1478

        >>> arr.map(pd.Series([10, 11, 12], index=[0, 1, 2]))
        <SparseArray>
        [10, 11, 12]
        Length: 3, dtype: Sparse[int64, np.int64(10)]
        """
        is_map = isinstance(mapper, (abc.Mapping, ABCSeries))

        fill_val = self.fill_value

        if na_action is None or notna(fill_val):
            fill_val = mapper.get(fill_val, fill_val) if is_map else mapper(fill_val)

        def func(sp_val):
            new_sp_val = mapper.get(sp_val, None) if is_map else mapper(sp_val)
            # check identity and equality because nans are not equal to each other
            if new_sp_val is fill_val or new_sp_val == fill_val:
                msg = "fill value in the sparse values not supported"
                raise ValueError(msg)
            return new_sp_val

        sp_values = [func(x) for x in self.sp_values]

        return type(self)(sp_values, sparse_index=self.sp_index, fill_value=fill_val)

    def _groupby_op(
        self,
        *,
        how: str,
        has_dropped_na: bool,
        min_count: int,
        ngroups: int,
        ids: npt.NDArray[np.intp],
        **kwargs,
    ):
        # first/last are handled by the base class to preserve EA type
        if how in ["first", "last"]:

View on GitHub (pinned to 3b7651241d)

Solutions

1. Change the array's fill_value to one the mapper cannot produce, e.g. SparseArray(data, fill_value=np.nan).
2. Map on the dense Series then re-sparse: pd.Series(np.asarray(arr)).map(m).astype(pd.SparseDtype()).
3. Adjust the mapper so it never returns the current fill_value for stored entries.

Example fix

// before
arr = pd.arrays.SparseArray([1.0, 2.0, 0.0], fill_value=0.0)
pd.Series(arr).map(lambda x: 0.0 if x > 5 else x)  # raises
// after
mapped = pd.Series(np.asarray(arr)).map(lambda x: 0.0 if x > 5 else x)
arr = pd.arrays.SparseArray(mapped.to_numpy(), fill_value=0.0)

def safe_sparse_map(arr, mapper):
    import numpy as np, pandas as pd
    fv = arr.fill_value
    mapped = pd.Series(np.asarray(arr)).map(mapper)
    return pd.arrays.SparseArray(mapped.to_numpy(), fill_value=fv)

def mapper_can_emit_fill_value(mapper, fill_value) -> bool:
    # heuristic: test against a sample of stored values
    return any(mapper(v) == fill_value for v in [fill_value])

try:
    pd.Series(arr).map(mapper)
except ValueError as e:
    if 'fill value in the sparse values' in str(e):
        import numpy as np, pandas as pd
        out = pd.arrays.SparseArray(pd.Series(np.asarray(arr)).map(mapper).to_numpy(), fill_value=arr.fill_value)
    else:
        raise

When it happens

Trigger: pd.Series(sparse_arr).map(lambda x: fill_value); a dict mapper whose output coincides with the SparseArray fill_value; e.g. fill_value is 0.0 and the mapper returns 0.0 for some stored value.

Common situations: Using Series.map to clamp or replace values that land on the fill_value; default-dict mappers that fall back to the fill_value.

Related errors

• SparseArray does not support item assignment via setitem
• na_action must either be 'ignore' or None, {na_action} was p
• Cannot modify read-only array
• Cannot modify read-only array
• Can only use the '.sparse' accessor with Sparse data.

AI-assisted analysis of pandas-dev/pandas@3b7651241d (2026-08-11). Data as JSON: /api/errors/f098cc33eb3ee36b. Report an issue: GitHub.