{"record":{"id":"2aac6df0237e33ba","repo":"deepfakes/faceswap","slug":"not-enough-ram-available-to-sort-faces-try-reduci","errorCode":null,"errorMessage":"Not enough RAM available to sort faces. Try reducing the size of  your dataset. Free RAM: {int(free_ram)}MB. Required RAM: {int(vector_required)}MB","messagePattern":"Not enough RAM available to sort faces\\. Try reducing the size of  your dataset\\. Free RAM: (.+?)MB\\. Required RAM: (.+?)MB","errorType":"exception","errorClass":"FaceswapError","httpStatus":null,"severity":"error","filePath":"lib/infer/identity.py","lineNumber":574,"sourceCode":"        divider = 1024 * 1024  # bytes to MB\n\n        free_ram = psutil.virtual_memory().available / divider\n        linkage_required = (((self._num_predictions ** 2) * np_float) / 1.8) / divider\n        vector_required = ((self._num_predictions * dims) * np_float) / divider\n        logger.debug(\"free_ram: %sMB, linkage_required: %sMB, vector_required: %sMB\",\n                     int(free_ram), int(linkage_required), int(vector_required))\n\n        if linkage_required < free_ram:\n            logger.verbose(\"Using linkage method\")  # type:ignore[attr-defined]\n            retval = False\n        elif vector_required < free_ram:\n            logger.warning(\"Not enough RAM to perform linkage clustering. Using vector \"\n                           \"clustering. This will be significantly slower. Free RAM: %sMB. \"\n                           \"Required for linkage method: %sMB\",\n                           int(free_ram), int(linkage_required))\n            retval = True\n        else:\n            raise FaceswapError(\"Not enough RAM available to sort faces. Try reducing \"\n                                f\"the size of  your dataset. Free RAM: {int(free_ram)}MB. \"\n                                f\"Required RAM: {int(vector_required)}MB\")\n        logger.debug(retval)\n        return retval\n\n    def _do_linkage(self,\n                    predictions: np.ndarray,\n                    method: T.Literal[\"single\", \"centroid\", \"median\", \"ward\"]) -> np.ndarray:\n        \"\"\"Use FastCluster to perform vector or standard linkage\n\n        Parameters\n        ----------\n        predictions\n            A stacked matrix of identity predictions of the shape (`N`, `D`) where `N` is the\n            number of observations and `D` are the number of dimensions.\n        method\n            The clustering method to use.\n","sourceCodeStart":556,"sourceCodeEnd":592,"githubUrl":"https://github.com/deepfakes/faceswap/blob/f530cb7508ae670f6474f8a7d9c4df94705cf96b/lib/infer/identity.py#L556-L592","documentation":"In identity-based face sorting, faceswap estimates RAM needed for linkage clustering vs. vector clustering. If even the cheaper vector method's requirement exceeds free system RAM, this FaceswapError aborts the sort rather than swapping the machine to death.","triggerScenarios":"Running sort by identity on a very large face set (hundreds of thousands of embeddings) on a machine with insufficient free RAM; free RAM already consumed by caches or other processes.","commonSituations":"Sorting massive extracted datasets on 8-16GB machines; running sort right after extraction while caches are full; container memory limits lower than host RAM.","solutions":["Reduce the dataset size: split faces into subsets and sort each separately.","Free RAM: close other applications, drop caches, increase container/pod memory limit.","Move to a machine with more RAM for the final sort.","Retry after freeing memory — free_ram is measured at runtime."],"exampleFix":"# before\n$ python tools.py sort -i /faces -t identity -o /sorted\n# FaceswapError: Not enough RAM...\n\n# after (split then sort)\n$ split -n l/4 /faces /faces_part_\n$ for d in /faces_part_*; do python tools.py sort -i $d -t identity -o ${d}_sorted; done","handlingStrategy":"validation","validationCode":"import psutil, math\n\nfree_ram_mib = psutil.virtual_memory().available >> 20\nest_required_mib = (num_faces ** 2) * 8 >> 20  # vector method ~ N^2 float64\nif est_required_mib > free_mib:\n    raise SystemExit('dataset too large for in-RAM sort; split it first')","typeGuard":null,"tryCatchPattern":"try:\n    sort_by_identity(faces)\nexcept FaceswapError as err:\n    if 'Not enough RAM' in str(err):\n        for chunk in split(faces, n=4):\n            sort_by_identity(chunk)\n    else:\n        raise","preventionTips":["Estimate N^2 memory cost before sorting large face sets.","Free page cache and close apps before identity sorting.","Give containers a memory limit comfortably above the N^2 estimate."],"tags":["memory","sorting","clustering","dataset-size"],"backgroundTag":null,"analyzedSha":"f530cb7508ae670f6474f8a7d9c4df94705cf96b","analyzedAt":"2026-08-15T02:59:26.626Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}