pnpm/pnpm · error · TarballError::ReadLocalTarball

local tarball changed while reading; refused to read past {s

Error message

local tarball changed while reading; refused to read past {size} bytes

What it means

The local-tarball reader caps the read at the stat-reported size plus one byte (take(size+1)); if more than `size` bytes were actually read, the file must have grown between the stat that produced `size` and the read itself. This InvalidData error refuses to read past the recorded size, treating the file as modified concurrently (a TOCTOU guard) rather than returning a mix of old and new bytes.

Source

Thrown at pnpm/crates/tarball/src/local_tarball.rs:69

    size: u64,
) -> Result<Vec<u8>, TarballError> {
    use tokio::io::AsyncReadExt;

    let read_limit = size.checked_add(1).ok_or_else(|| {
        read_local_tarball_error(
            path,
            io::ErrorKind::InvalidData,
            format!("local tarball is too large to read into memory ({size} bytes)"),
        )
    })?;
    let mut buffer = allocate_local_tarball_buffer(path, package_url, size)?;
    let mut reader = file.take(read_limit);
    reader
        .read_to_end(&mut buffer)
        .await
        .map_err(|source| TarballError::ReadLocalTarball { path: path.to_path_buf(), source })?;
    if u64::try_from(buffer.len()).unwrap_or(u64::MAX) > size {
        return Err(read_local_tarball_error(
            path,
            io::ErrorKind::InvalidData,
            format!("local tarball changed while reading; refused to read past {size} bytes"),
        ));
    }
    Ok(buffer)
}

pub(crate) fn allocate_local_tarball_buffer(
    path: &Path,
    package_url: &str,
    size: u64,
) -> Result<Vec<u8>, TarballError> {
    allocate_tarball_buffer(Some(size), package_url).map_err(|error| match error {
        TarballError::TarballTooLarge { .. } => read_local_tarball_error(
            path,
            io::ErrorKind::InvalidData,
            format!("local tarball is too large to read into memory ({size} bytes)"),

View on GitHub (pinned to 6261b7f388)

Solutions

1. Finish generating/re-packing the tarball before starting the install (add a dependency between the build and install steps)
2. Retry the install after the file has settled
3. Write new artifacts to versioned/immutable filenames instead of overwriting one path in place

Example fix

# before (CI): pack and install race
- run: pnpm -C libs/ui pack & pnpm install

# after: sequenced
- run: pnpm -C libs/ui pack
- run: pnpm install

fn tarball_stable(path: &Path) -> bool {
    let a = std::fs::metadata(path).map(|m| (m.len(), m.modified().ok()));
    std::thread::sleep(std::time::Duration::from_millis(50));
    let b = std::fs::metadata(path).map(|m| (m.len(), m.modified().ok()));
    a.is_ok() && a == b
}

fn is_changed_during_read(err: &TarballError) -> bool {
    matches!(err, TarballError::ReadLocalTarball { source, .. }
        if source.kind() == std::io::ErrorKind::InvalidData
            && source.to_string().contains("changed while reading"))
}

match read_local_tarball(path).await {
    Err(e) if is_changed_during_read(&e) => read_local_tarball(path).await, // re-stat + re-read once settled
    other => other,
}

When it happens

Trigger: The tarball is rewritten while pnpm reads it: a build step re-packing the .tgz during install, a CI artifact still being written when install starts, or a watch loop regenerating the file.

Common situations: CI pipelines where packaging and install stages race; local dev with a pack-on-save watcher; artifact synced over network while consumed.

Related errors

• local tarball path is not a regular file
• local tarball is too large to read into memory ({size} bytes
• MISSING_PACKAGE_JSON
• tar entry path rejected (non-normal component, possible dire
• tar entry path has no payload after dropping the top-level c

AI-assisted analysis of pnpm/pnpm@6261b7f388 (2026-08-17). Data as JSON: /api/errors/c0bfedcf2b4530f2. Report an issue: GitHub.