{"record":{"id":"f098cc33eb3ee36b","repo":"pandas-dev/pandas","slug":"fill-value-in-the-sparse-values-not-supported","errorCode":null,"errorMessage":"fill value in the sparse values not supported","messagePattern":"fill value in the sparse values not supported","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"pandas/core/arrays/sparse/array.py","lineNumber":1478,"sourceCode":"\n        >>> arr.map(pd.Series([10, 11, 12], index=[0, 1, 2]))\n        <SparseArray>\n        [10, 11, 12]\n        Length: 3, dtype: Sparse[int64, np.int64(10)]\n        \"\"\"\n        is_map = isinstance(mapper, (abc.Mapping, ABCSeries))\n\n        fill_val = self.fill_value\n\n        if na_action is None or notna(fill_val):\n            fill_val = mapper.get(fill_val, fill_val) if is_map else mapper(fill_val)\n\n        def func(sp_val):\n            new_sp_val = mapper.get(sp_val, None) if is_map else mapper(sp_val)\n            # check identity and equality because nans are not equal to each other\n            if new_sp_val is fill_val or new_sp_val == fill_val:\n                msg = \"fill value in the sparse values not supported\"\n                raise ValueError(msg)\n            return new_sp_val\n\n        sp_values = [func(x) for x in self.sp_values]\n\n        return type(self)(sp_values, sparse_index=self.sp_index, fill_value=fill_val)\n\n    def _groupby_op(\n        self,\n        *,\n        how: str,\n        has_dropped_na: bool,\n        min_count: int,\n        ngroups: int,\n        ids: npt.NDArray[np.intp],\n        **kwargs,\n    ):\n        # first/last are handled by the base class to preserve EA type\n        if how in [\"first\", \"last\"]:","sourceCodeStart":1460,"sourceCodeEnd":1496,"githubUrl":"https://github.com/pandas-dev/pandas/blob/3b7651241d4da534b3559b60ef128e1c34f54116/pandas/core/arrays/sparse/array.py#L1460-L1496","documentation":"ValueError from SparseArray mapping (used by Series.map/.apply on sparse-backed Series) when a mapped sparse value would equal the array's fill_value. Allowing it would corrupt the sparse/dense distinction because the fill_value is implicit and not stored in sp_values.","triggerScenarios":"pd.Series(sparse_arr).map(lambda x: fill_value); a dict mapper whose output coincides with the SparseArray fill_value; e.g. fill_value is 0.0 and the mapper returns 0.0 for some stored value.","commonSituations":"Using Series.map to clamp or replace values that land on the fill_value; default-dict mappers that fall back to the fill_value.","solutions":["Change the array's fill_value to one the mapper cannot produce, e.g. SparseArray(data, fill_value=np.nan).","Map on the dense Series then re-sparse: pd.Series(np.asarray(arr)).map(m).astype(pd.SparseDtype()).","Adjust the mapper so it never returns the current fill_value for stored entries."],"exampleFix":"// before\narr = pd.arrays.SparseArray([1.0, 2.0, 0.0], fill_value=0.0)\npd.Series(arr).map(lambda x: 0.0 if x > 5 else x)  # raises\n// after\nmapped = pd.Series(np.asarray(arr)).map(lambda x: 0.0 if x > 5 else x)\narr = pd.arrays.SparseArray(mapped.to_numpy(), fill_value=0.0)","handlingStrategy":"validation","validationCode":"def safe_sparse_map(arr, mapper):\n    import numpy as np, pandas as pd\n    fv = arr.fill_value\n    mapped = pd.Series(np.asarray(arr)).map(mapper)\n    return pd.arrays.SparseArray(mapped.to_numpy(), fill_value=fv)","typeGuard":"def mapper_can_emit_fill_value(mapper, fill_value) -> bool:\n    # heuristic: test against a sample of stored values\n    return any(mapper(v) == fill_value for v in [fill_value])","tryCatchPattern":"try:\n    pd.Series(arr).map(mapper)\nexcept ValueError as e:\n    if 'fill value in the sparse values' in str(e):\n        import numpy as np, pandas as pd\n        out = pd.arrays.SparseArray(pd.Series(np.asarray(arr)).map(mapper).to_numpy(), fill_value=arr.fill_value)\n    else:\n        raise","preventionTips":["Choose a fill_value the mapper cannot produce (e.g. np.nan).","Map on dense Series then re-sparse.","Audit mapper outputs against the current fill_value."],"tags":["sparse","map","fill-value","mutation"],"backgroundTag":null,"analyzedSha":"3b7651241d4da534b3559b60ef128e1c34f54116","analyzedAt":"2026-08-11T22:10:44.015Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}