sgl-project/sglang · error · ValueError

tar material payload is truncated with {remaining} bytes lef

Error message

tar material payload is truncated with {remaining} bytes left

What it means

While streaming the member bytes, read() returned empty before the declared size was consumed — the file shrank or hit EOF mid-read despite passing the earlier stat check.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/minimax_h3/material_io.py:685

        raise ValueError(
            f"tar material payload is truncated: expected {size} bytes, "
            f"only {available} available"
        )

    output_path, partial_path = _material_output_paths(
        batch,
        condition_type=condition_type,
        condition_index=condition_index,
        source_name=member,
    )
    remaining = size
    try:
        with source_path.open("rb") as source, partial_path.open("wb") as output:
            source.seek(offset)
            while remaining:
                chunk = source.read(min(MINIMAX_H3_HTTP_READ_CHUNK_BYTES, remaining))
                if not chunk:
                    raise ValueError(
                        f"tar material payload is truncated with {remaining} bytes left"
                    )
                if len(chunk) > remaining:
                    raise ValueError("tar material reader returned too many bytes")
                output.write(chunk)
                remaining -= len(chunk)
        partial_path.replace(output_path)
    except Exception:
        partial_path.unlink(missing_ok=True)
        output_path.unlink(missing_ok=True)
        raise
    return str(output_path)


def _http_media_type(response: Any) -> str | None:
    headers = getattr(response, "headers", None)
    if headers is None:
        return None

View on GitHub (pinned to 0132848349)

Solutions

1. Wait for archive writes to complete before serving requests (write to temp then rename)
2. Re-verify file size with checksums before the run
3. Pin a stable snapshot/immutable path for the archive

if source.stat().st_size < offset + size: wait_or_repair()

except ValueError as e:
    if 'truncated with' in str(e): retry_after_archive_settles()

When it happens

Trigger: The tar file is truncated or concurrently modified between the stat check and the streaming read loop.

Common situations: Archive still being written when the request arrives; NFS/volume flakiness; disk-full truncation.

Related errors

• tar material payload is truncated: expected {size} bytes, on
• Cannot transition {request_id} from terminal state {old_stat
• tar material offset_data and size must be non-negative
• tar material payload is empty
• tar material source does not exist or is not a file: {source

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ed6b9d0a96b1ca1c. Report an issue: GitHub.