{"record":{"id":"2b363a3d85e9f266","repo":"unslothai/unsloth","slug":"the-pinned-spec-name-archive-contains-duplicate","errorCode":null,"errorMessage":"The pinned {spec.name} archive contains duplicate files","messagePattern":"The pinned (.+?) archive contains duplicate files","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"studio/backend/utils/third_party_source.py","lineNumber":670,"sourceCode":"                                )\n                            parts = _archive_member_parts(member, spec)\n                            if member.isdir():\n                                continue\n                            if not member.isfile() or member.size < 0:\n                                raise RuntimeError(\n                                    f\"The pinned {spec.name} archive contains a non-regular file\"\n                                )\n                            uncompressed_bytes += member.size\n                            if uncompressed_bytes > _ARCHIVE_MAX_UNCOMPRESSED_BYTES:\n                                raise RuntimeError(\n                                    f\"The pinned {spec.name} archive expands too large\"\n                                )\n                            if len(parts) < 3 or parts[1] != spec.package:\n                                continue\n                            relative = \"/\".join(parts[1:])\n                            _package_path_parts(relative, spec, kind = \"archive\")\n                            if relative in extracted:\n                                raise RuntimeError(\n                                    f\"The pinned {spec.name} archive contains duplicate files\"\n                                )\n                            extracted.add(relative)\n                            source_file = bundle.extractfile(member)\n                            if source_file is None:\n                                raise RuntimeError(\n                                    f\"The pinned {spec.name} archive contains an unreadable file\"\n                                )\n                            destination_file = staging.joinpath(*parts[1:])\n                            destination_file.parent.mkdir(parents = True, exist_ok = True)\n                            remaining = member.size\n                            with source_file, destination_file.open(\"wb\") as handle:\n                                while remaining:\n                                    chunk = source_file.read(min(1024 * 1024, remaining))\n                                    if not chunk:\n                                        raise RuntimeError(\n                                            f\"The pinned {spec.name} archive ended unexpectedly\"\n                                        )","sourceCodeStart":652,"sourceCodeEnd":688,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/utils/third_party_source.py#L652-L688","documentation":"The installer records each extracted relative path in a set; seeing the same relative path (under the package subtree) twice raises this error. Duplicate entries are a known tar smuggling technique — a benign file passes review, then a second entry with the same name overwrites it during extraction — so the library refuses rather than letting later entries win.","triggerScenarios":"Two tar members normalize to the same `package/...` relative path — e.g. `repo-sha/foo/__init__.py` appearing twice, or case/directory variants that collapse to identical extracted paths within the parts[1]==spec.package subtree.","commonSituations":"Repacked archives accidentally containing a file twice (some zip->tar converters do this); maliciously crafted archives with a trailing duplicate overwrite entry; unusual archivers that emit both a dir-prefixed and bare variant of the same file.","solutions":["Find duplicates: tar -tzf source.tar.gz | sort | uniq -d","Repack the tree cleanly (fresh export from the tagged revision) and re-pin digests","Always pull from the canonical codeload URL for the revision rather than ad-hoc repacked mirrors","If canonical artifacts duplicate, the pin itself is unusable — pin a different revision"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"import tarfile, collections\nwith tarfile.open(\"source.tar.gz\") as tf:\n    names = [m.name for m in tf]\ndups = [n for n, c in collections.Counter(names).items() if c > 1]\nassert not dups, f\"duplicate entries: {dups}\"","typeGuard":"def archive_has_no_duplicates(archive: Path) -> bool:\n    with tarfile.open(archive) as tf:\n        seen = set()\n        for m in tf:\n            if m.name in seen:\n                return False\n            seen.add(m.name)\n    return True","tryCatchPattern":"try:\n    ensure_pinned_source(spec)\nexcept RuntimeError as e:\n    if \"duplicate files\" in str(e):\n        raise SystemExit(\"archive contains duplicate entries — repin to a clean export\")","preventionTips":["Always export fresh from the tagged revision with standard tar/git archive; do not merge or append to existing tarballs","Treat duplicate entries as a supply-chain red flag: verify the artifact digest before investigating further"],"tags":["security","tarfile","duplicate-entries","extraction","supply-chain"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}