{"record":{"id":"eaede111f49e5ba4","repo":"pandas-dev/pandas","slug":"values-should-be-unique-if-codes-is-not-none","errorCode":null,"errorMessage":"values should be unique if codes is not None","messagePattern":"values should be unique if codes is not None","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"pandas/core/algorithms.py","lineNumber":1752,"sourceCode":"    codes = ensure_platform_int(np.asarray(codes))\n\n    # ranks[i] gives the position of values[i] in `ordered`\n    if use_counting:\n        arr = cast(\"np.ndarray\", values)\n        if arr.dtype.kind == \"i\":\n            # go through int64 so differences don't overflow narrower signed\n            #  dtypes; int64 wraparound is exact since the true differences\n            #  are within rng_size\n            shifted = arr.astype(np.int64, copy=False) - vmin\n        else:\n            # unsigned: differences always fit the unsigned dtype\n            shifted = arr - vmin\n        present = np.zeros(rng_size, dtype=bool)\n        present[shifted] = True\n        counts = present.cumsum(dtype=np.intp)\n        # the counting pass gives the uniqueness check for free\n        if not assume_unique and counts[-1] != len(values):\n            raise ValueError(\"values should be unique if codes is not None\")\n        ranks = counts[shifted]\n        ranks -= 1\n    else:\n        if not assume_unique and not len(unique(values)) == len(values):\n            raise ValueError(\"values should be unique if codes is not None\")\n\n        if sorter is not None:\n            # sorter is a permutation, so scatter is a faster equivalent to\n            #  `ranks = sorter.argsort()`\n            ranks = np.empty(len(sorter), dtype=np.intp)\n            ranks[sorter] = np.arange(len(sorter), dtype=np.intp)\n        else:\n            # mixed types\n            # error: Argument 1 to \"_get_hashtable_algo\" has incompatible type\n            # \"Union[Index, ExtensionArray, ndarray[Any, Any]]\"; expected\n            # \"ndarray[Any, Any]\"\n            hash_klass, values = _get_hashtable_algo(values)  # type: ignore[arg-type]\n            t = hash_klass(len(values))","sourceCodeStart":1734,"sourceCodeEnd":1770,"githubUrl":"https://github.com/pandas-dev/pandas/blob/71959b8cb9b2459c16e14b34f28b178ccfe14735/pandas/core/algorithms.py#L1734-L1770","documentation":"Raised by safe_sort on the fast counting-sort path: when codes is not None and assume_unique is False, pandas verifies values are unique by comparing the counting-sort coverage to len(values). If values contain duplicates the code remap would be ambiguous, so a ValueError is raised. The message notes uniqueness is required only when codes is provided.","triggerScenarios":"safe_sort(np.array([1, 1, 2]), codes=[0, 1, 2]) on an integer array large enough to trigger counting sort, with duplicate values present; factorize/unique pipelines that feed duplicate values plus codes into safe_sort.","commonSituations":"Pre-aggregated data where the 'values' index accidentally contains repeats; merging arrays that introduced duplicates before sorting; passing assume_unique=False (default) on data that still has dups.","solutions":["Deduplicate values before calling safe_sort (e.g. np.unique).","If values really are unique, pass assume_unique=True to skip the check (counting path).","Pass codes=None if you only need the sorted values."],"exampleFix":"# before\nsafe_sort(np.array([1, 1, 2, 3]), codes=[0, 1, 2])\n# after\nvals, idx = np.unique([1, 1, 2, 3], return_index=True)\nsafe_sort(vals, codes=idx, assume_unique=True)","handlingStrategy":"validation","validationCode":"import numpy as np\n\ndef safe_sort_unique(values, codes, assume_unique=False):\n    if codes is not None and not assume_unique:\n        if len(np.unique(values)) != len(values):\n            values, inv = np.unique(values, return_inverse=True)\n            codes = inv[np.asarray(codes)]\n            assume_unique = True\n    return pd.core.algorithms.safe_sort(values, codes, assume_unique=assume_unique)","typeGuard":"import numpy as np\n\ndef values_are_unique(values) -> bool:\n    v = np.asarray(values)\n    return len(np.unique(v)) == len(v)","tryCatchPattern":null,"preventionTips":["Deduplicate values (np.unique) before passing codes to safe_sort.","Pass assume_unique=True only when uniqueness is guaranteed.","Use codes=None if you only need sorted values."],"tags":["safe-sort","unique","codes","validation"],"analyzedSha":"71959b8cb9b2459c16e14b34f28b178ccfe14735","analyzedAt":"2026-08-07T01:30:20.476Z","schemaVersion":2},"datasetVersion":"2026-08-07T03:17:09.362Z"}