{"record":{"id":"c5d974aad93ec3c0","repo":"pandas-dev/pandas","slug":"the-index-columns-must-be-unique-when-raw-false-an","errorCode":null,"errorMessage":"The index/columns must be unique when raw=False and engine='numba'","messagePattern":"The index/columns must be unique when raw=False and engine='numba'","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"pandas/core/apply.py","lineNumber":1311,"sourceCode":"\n        results = {}\n\n        for i, v in enumerate(series_gen):\n            results[i] = self.func(v, *self.args, **self.kwargs)\n            if isinstance(results[i], ABCSeries):\n                # If we have a view on v, we need to make a copy because\n                #  series_generator will swap out the underlying data\n                results[i] = results[i].copy(deep=False)\n\n        return results, res_index\n\n    def apply_series_numba(self):\n        if self.engine_kwargs.get(\"parallel\", False):\n            raise NotImplementedError(\n                \"Parallel apply is not supported when raw=False and engine='numba'\"\n            )\n        if not self.obj.index.is_unique or not self.columns.is_unique:\n            raise NotImplementedError(\n                \"The index/columns must be unique when raw=False and engine='numba'\"\n            )\n        self.validate_values_for_numba()\n        results = self.apply_with_numba()\n        return results, self.result_index\n\n    def wrap_results(self, results: ResType, res_index: Index) -> DataFrame | Series:\n        from pandas import Series\n\n        # see if we can infer the results\n        if len(results) > 0 and 0 in results and is_sequence(results[0]):\n            return self.wrap_results_for_axis(results, res_index)\n\n        # dict of scalars\n\n        # the default dtype of an empty Series is `object`, but this\n        # code can be hit by df.mean() where the result should have dtype\n        # float64 even if it's an empty Series.","sourceCodeStart":1293,"sourceCodeEnd":1329,"githubUrl":"https://github.com/pandas-dev/pandas/blob/71959b8cb9b2459c16e14b34f28b178ccfe14735/pandas/core/apply.py#L1293-L1329","documentation":"Raised by apply_series_numba (apply.py:1310-1313) when engine='numba' is used with raw=False on a frame whose index or columns are not unique. The numba Series-passing path assembles results by positionally re-attaching them to the existing index/columns, which is only unambiguous when labels are unique; duplicate labels would make the result ambiguous, so pandas requires uniqueness up front.","triggerScenarios":"df.apply(func, engine='numba') (raw defaults to False) on a DataFrame with duplicate index labels or duplicate column names. Triggered at apply.py:1310-1313 when self.obj.index.is_unique or self.columns.is_unique is False.","commonSituations":"Frames built from concat/merge/join operations that retained duplicate labels; CSVs whose key column has dupes used as the index; intentionally duplicated columns from a pivot or concatenation step; transitioning a workflow to engine='numba' without de-duplicating labels.","solutions":["Reset or deduplicate the index before applying: df = df.reset_index(drop=True).","Deduplicate columns by renaming or dropping dupes: df.columns = pd.io.parsers ParserBase... or df.loc[:, ~df.columns.duplicated()].","Switch to raw=True (the raw numba path does not require label uniqueness).","Fall back to engine='python' if label uniqueness must be preserved."],"exampleFix":"// before\ndf.apply(func, engine='numba')  # df has duplicate index values\n// after\ndf = df.reset_index(drop=True)\ndf.apply(func, engine='numba')","handlingStrategy":"validation","validationCode":"if engine == 'numba' and raw is False:\n    if not df.index.is_unique:\n        raise ValueError('numba engine requires a unique index; call reset_index(drop=True)')\n    if hasattr(df, 'columns') and not df.columns.is_unique:\n        raise ValueError('numba engine requires unique column names')","typeGuard":"def safe_for_numba_series_apply(df, engine: str, raw: bool) -> bool:\n    if engine != 'numba' or raw:\n        return True\n    idx_ok = getattr(df, 'index', None) is None or df.index.is_unique\n    col_ok = not hasattr(df, 'columns') or df.columns.is_unique\n    return idx_ok and col_ok","tryCatchPattern":"try:\n    df.apply(func, engine='numba')\nexcept NotImplementedError as e:\n    if 'index/columns must be unique' in str(e).lower():\n        df.reset_index(drop=True).apply(func, engine='numba')\n    else:\n        raise","preventionTips":["De-duplicate the index/columns before any numba-engine apply.","Add a uniqueness assert in your data-loading pipeline to fail early."],"tags":["pandas","numba","apply","duplicate-index","engine"],"analyzedSha":"71959b8cb9b2459c16e14b34f28b178ccfe14735","analyzedAt":"2026-08-07T01:30:20.476Z","schemaVersion":2},"datasetVersion":"2026-08-07T03:17:09.362Z"}