{"record":{"id":"3c0a5da3f89313e9","repo":"unslothai/unsloth","slug":"the-pinned-source-archive-expands-too-large","errorCode":null,"errorMessage":"The pinned source archive expands too large","messagePattern":"The pinned source archive expands too large","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"studio/backend/utils/third_party_source.py","lineNumber":626,"sourceCode":"        or parts[0] != _archive_root_name(spec)\n    ):\n        raise RuntimeError(f\"Invalid path in the pinned {spec.name} source archive\")\n    return parts\n\n\nclass _BoundedArchiveReader:\n    def __init__(self, handle, limit: int):\n        self._handle = handle\n        self._limit = limit\n        self._read = 0\n\n    def read(self, size: int = -1) -> bytes:\n        remaining = self._limit - self._read\n        requested = remaining + 1 if size < 0 else min(size, remaining + 1)\n        data = self._handle.read(requested)\n        self._read += len(data)\n        if self._read > self._limit:\n            raise RuntimeError(\"The pinned source archive expands too large\")\n        return data\n\n\ndef _install_archive_source(destination: Path, spec: PinnedSource) -> None:\n    if spec.archive_url is None or spec.source_tree_digest is None:\n        raise RuntimeError(f\"The pinned {spec.name} source archive is not configured\")\n    workspace = Path(tempfile.mkdtemp(prefix = \".archive-\", dir = destination.parent))\n    archive = workspace / \"source.tar.gz\"\n    staging = workspace / \"source\"\n    staging.mkdir()\n    try:\n        _download_archive(spec.archive_url, archive, spec)\n        member_count = 0\n        uncompressed_bytes = 0\n        extracted = set()\n        try:\n            with archive.open(\"rb\") as compressed:\n                with gzip.GzipFile(fileobj = compressed, mode = \"rb\") as decompressed:","sourceCodeStart":608,"sourceCodeEnd":644,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/utils/third_party_source.py#L608-L644","documentation":"Raised by _BoundedArchiveReader.read when cumulative uncompressed bytes fed to tarfile exceed _ARCHIVE_MAX_TAR_BYTES. The reader wraps the gzip stream and only ever exposes limit+1 bytes, so a gzip bomb is detected as soon as the decompressed stream crosses the cap rather than after decompressing gigabytes.","triggerScenarios":"The downloaded .tar.gz decompresses to more than _ARCHIVE_MAX_TAR_BYTES: a genuine gzip bomb (highly compressible filler) or a legitimately huge source tree exceeding the built-in uncompressed budget.","commonSituations":"Malicious archive substitution at the URL (defense triggers, install aborts); a dependency that vendored enormous generated files/test data so its source tree exceeds the cap; mis-pinned archive containing a full git history bundle.","solutions":["Verify the artifact out-of-band: gzip -dc source.tar.gz | wc -c and compare with _ARCHIVE_MAX_TAR_BYTES; check its SHA-256 against spec.source_tree_digest","If the pin digest matches and the tree is legitimately that large, raise _ARCHIVE_MAX_TAR_BYTES (and _ARCHIVE_MAX_UNCOMPRESSED_BYTES) in your build","If the digest does not match, the URL serves tampered content — repin to the canonical source and treat as a security incident","Prefer slim source tarballs over bundles that include history or binaries"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"import gzip\nuncompressed = 0\nwith gzip.open(\"source.tar.gz\", \"rb\") as f:\n    while True:\n        chunk = f.read(1024 * 1024)\n        if not chunk:\n            break\n        uncompressed += len(chunk)\n        if uncompressed > _ARCHIVE_MAX_TAR_BYTES:\n            raise SystemExit(\"decompressed tar stream exceeds cap\")","typeGuard":null,"tryCatchPattern":"try:\n    ensure_pinned_source(spec)\nexcept RuntimeError as e:\n    if \"expands too large\" in str(e):\n        # verify digest; if pin is authentic and tree is legitimately big, adjust cap in your build","preventionTips":["Record decompressed size in CI metadata for each pin so cap regressions surface at pin time, not runtime","Prefer subtree exports over full snapshots to keep decompressed size far below the cap"],"tags":["security","decompression-bomb","gzip","tarfile","resource-limit"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}