unslothai/unsloth · error · RuntimeError

The pinned {spec.name} archive expands too large

Error message

The pinned {spec.name} archive expands too large

What it means

Running sum of every member.size (for regular files) is compared against _ARCHIVE_MAX_UNCOMPRESSED_BYTES; exceeding it aborts extraction. Distinct from the 1586 cap (which bounds raw decompressed TAR stream bytes via _BoundedArchiveReader, _ARCHIVE_MAX_TAR_BYTES), this bounds the declared file payload that would actually land on disk.

Source

Thrown at studio/backend/utils/third_party_source.py:662

                with gzip.GzipFile(fileobj = compressed, mode = "rb") as decompressed:
                    reader = _BoundedArchiveReader(decompressed, _ARCHIVE_MAX_TAR_BYTES)
                    with tarfile.open(fileobj = reader, mode = "r|") as bundle:
                        for member in bundle:
                            member_count += 1
                            if member_count > _ARCHIVE_MAX_MEMBERS:
                                raise RuntimeError(
                                    f"The pinned {spec.name} archive has too many entries"
                                )
                            parts = _archive_member_parts(member, spec)
                            if member.isdir():
                                continue
                            if not member.isfile() or member.size < 0:
                                raise RuntimeError(
                                    f"The pinned {spec.name} archive contains a non-regular file"
                                )
                            uncompressed_bytes += member.size
                            if uncompressed_bytes > _ARCHIVE_MAX_UNCOMPRESSED_BYTES:
                                raise RuntimeError(
                                    f"The pinned {spec.name} archive expands too large"
                                )
                            if len(parts) < 3 or parts[1] != spec.package:
                                continue
                            relative = "/".join(parts[1:])
                            _package_path_parts(relative, spec, kind = "archive")
                            if relative in extracted:
                                raise RuntimeError(
                                    f"The pinned {spec.name} archive contains duplicate files"
                                )
                            extracted.add(relative)
                            source_file = bundle.extractfile(member)
                            if source_file is None:
                                raise RuntimeError(
                                    f"The pinned {spec.name} archive contains an unreadable file"
                                )
                            destination_file = staging.joinpath(*parts[1:])
                            destination_file.parent.mkdir(parents = True, exist_ok = True)

View on GitHub (pinned to 203007d190)

Solutions

  1. Estimate extracted size: tar -tzvf source.tar.gz | awk '{s+=$3} END {print s}' and compare with _ARCHIVE_MAX_UNCOMPRESSED_BYTES
  2. Repin to a subtree export containing only spec.package's files instead of the whole snapshot
  3. If legitimate, raise _ARCHIVE_MAX_UNCOMPRESSED_BYTES (and companion caps) in your build after checking available disk
  4. Confirm the digest matches the pin to rule out a crafted header attack
Defensive patterns

Strategy: validation

Validate before calling

import tarfile
total = 0
with tarfile.open("source.tar.gz") as tf:
    for m in tf:
        if m.isfile():
            total += m.size
assert total <= _ARCHIVE_MAX_UNCOMPRESSED_BYTES, f"payload {total} exceeds disk cap"

Try / catch

try:
    ensure_pinned_source(spec)
except RuntimeError as e:
    if "expands too large" in str(e):
        # distinguish via pre-checks which cap tripped; trim the export or adjust the constant

Prevention

When it happens

Trigger: Sum of member.size across all regular-file entries in the archive exceeds _ARCHIVE_MAX_UNCOMPRESSED_BYTES while iterating — e.g. a legitimate tree larger than the on-disk budget, or headers declaring huge sizes (the copy loop would attempt to write member.size bytes per file).

Common situations: Vendoring large binary assets in the dependency's repo; generated-data-heavy packages; mismatch between _ARCHIVE_MAX_TAR_BYTES and this tighter on-disk cap causing big-but-legal archives to fail here.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/d15308570cf416b2. Report an issue: GitHub.