jdx/mise · error

remote cache blob pack ended before a blob was complete

Error message

remote cache blob pack ended before a blob was complete

What it means

While streaming a blob's payload, mise-cache-core reads exactly `digest.size` bytes declared in the entry header. If the underlying stream returns 0 bytes (a clean EOF, since I/O errors surface as Err) before the declared size is reached, the pack is truncated and this error aborts the download. It means the server's framing promised more bytes than its response body actually contained. Note the pack download already runs inside the crate's retry loop, but this bail! is not classified transient, so it surfaces on the first occurrence.

Source

Thrown at crates/mise-cache-core/src/lib.rs:951

            bail!("remote cache blob pack returned a duplicate digest");
        }
        framed_bytes = framed_bytes
            .checked_add(BLOB_PACK_HEADER_BYTES)
            .and_then(|bytes| bytes.checked_add(digest.size))
            .ok_or_else(|| eyre!("remote cache blob pack is too large"))?;
        payload_bytes = payload_bytes
            .checked_add(digest.size)
            .ok_or_else(|| eyre!("remote cache blob pack payload is too large"))?;

        let path = directory.path().join(blobs.len().to_string());
        let mut output = tokio::fs::File::create(&path).await?;
        let mut remaining = digest.size;
        let mut buffer = [0_u8; 64 * 1024];
        while remaining > 0 {
            let limit = usize::try_from(remaining.min(buffer.len() as u64)).unwrap();
            let count = reader.read(&mut buffer[..limit]).await?;
            if count == 0 {
                bail!("remote cache blob pack ended before a blob was complete");
            }
            output.write_all(&buffer[..count]).await?;
            hasher.update(&buffer[..count]);
            remaining -= count as u64;
        }
        output.flush().await?;
        drop(output);
        if !hasher.matches(&digest.hash) {
            bail!("remote cache blob pack failed digest verification");
        }
        blobs.push((digest, path));
    }
    let blob_count = blobs.len().try_into().unwrap_or(u64::MAX);
    let metadata = metadata.validate(BlobPackResponseStats {
        blob_count,
        payload_bytes,
        framed_bytes,
    })?;

View on GitHub (pinned to 6f52dcdf99)

Solutions

  1. Retry the fetch once: truncation is often a one-off connection cut.
  2. Fix the server's pack writer to stream exactly `size` payload bytes after each header and flush before finishing the response; never skip a blob after emitting its header.
  3. Verify no proxy between client and server truncates streaming responses or enforces a max body size smaller than your blobs.
  4. Compare the response framing (Content-Length or chunk sizes) with the sum of entry headers to find which entry is short.
Defensive patterns

Strategy: retry

Try / catch

let pack = download_with_retry(|| remote_cache.download(staging_dir)).await;
// decode_blob_pack's bails are NOT retried by the crate's own retry loop,
// so wrap the fetch: on "ended before a blob was complete", back off briefly
// and re-request once; on a second failure, rebuild the artifacts locally.

Prevention

When it happens

Trigger: A blob pack entry header declares size N but the response body ends after fewer bytes: the server wrote the header then failed to flush or skipped a short blob, a gateway truncated the body (Content-Length mismatch), or the connection closed mid-stream in a way that surfaced as EOF rather than an error.

Common situations: Server-side pack writers that write the size field before serializing the blob and then skip missing blobs; reverse proxies truncating large streaming responses; server buffer limits smaller than the pack being produced.

Related errors

AI-assisted analysis of jdx/mise@6f52dcdf99 (2026-08-22). Data as JSON: /api/errors/fa7cd3530704c2a2. Report an issue: GitHub.