{"record":{"id":"6e94d300a11b1e28","repo":"pnpm/pnpm","slug":"local-tarball-is-too-large-to-read-into-memory-s","errorCode":null,"errorMessage":"local tarball is too large to read into memory ({size} bytes)","messagePattern":"local tarball is too large to read into memory \\((.+?) bytes\\)","errorType":"exception","errorClass":"TarballError::ReadLocalTarball","httpStatus":null,"severity":"error","filePath":"pnpm/crates/tarball/src/local_tarball.rs","lineNumber":56,"sourceCode":"        return Ok(());\n    }\n    Err(read_local_tarball_error(\n        path,\n        io::ErrorKind::InvalidInput,\n        \"local tarball path is not a regular file\",\n    ))\n}\n\npub(crate) async fn read_local_tarball_buffer(\n    file: tokio::fs::File,\n    path: &Path,\n    package_url: &str,\n    size: u64,\n) -> Result<Vec<u8>, TarballError> {\n    use tokio::io::AsyncReadExt;\n\n    let read_limit = size.checked_add(1).ok_or_else(|| {\n        read_local_tarball_error(\n            path,\n            io::ErrorKind::InvalidData,\n            format!(\"local tarball is too large to read into memory ({size} bytes)\"),\n        )\n    })?;\n    let mut buffer = allocate_local_tarball_buffer(path, package_url, size)?;\n    let mut reader = file.take(read_limit);\n    reader\n        .read_to_end(&mut buffer)\n        .await\n        .map_err(|source| TarballError::ReadLocalTarball { path: path.to_path_buf(), source })?;\n    if u64::try_from(buffer.len()).unwrap_or(u64::MAX) > size {\n        return Err(read_local_tarball_error(\n            path,\n            io::ErrorKind::InvalidData,\n            format!(\"local tarball changed while reading; refused to read past {size} bytes\"),\n        ));\n    }","sourceCodeStart":38,"sourceCodeEnd":74,"githubUrl":"https://github.com/pnpm/pnpm/blob/6261b7f388016d57ca6b90340342411cd1d0d00f/pnpm/crates/tarball/src/local_tarball.rs#L38-L74","documentation":"read_local_tarball_buffer pre-reserves size+1 bytes so it can detect a file that grew during the read; the size.checked_add(1) guard fires only when the stat-reported size is u64::MAX, i.e. the incremented bound cannot be represented. The message surfaces this as the tarball being too large to read into memory. It is unreachable for any real archive size - only a filesystem reporting an absurd st_size (broken FUSE/procfs-style mounts) can trip it.","triggerScenarios":"A file: dependency whose path stat()s with st_size == u64::MAX - pseudo-filesystems or faulty FUSE mounts reporting junk sizes for regular-looking files.","commonSituations":"Pointing a file: dependency at a special file on a pseudo-filesystem; container mounts with broken metadata.","solutions":["Check what the path really is: stat the file and confirm a sane size","Copy/recreate the tarball onto a normal filesystem and reference that copy","If a FUSE/network mount reports bogus sizes, read through the mount's own tooling or remount properly"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"fn sane_tarball_size(meta: &std::fs::Metadata) -> bool {\n    meta.len() != u64::MAX && usize::try_from(meta.len()).is_ok()\n}","typeGuard":"fn is_size_overflow(err: &TarballError) -> bool {\n    matches!(err, TarballError::ReadLocalTarball { source, .. }\n        if source.kind() == std::io::ErrorKind::InvalidData\n            && source.to_string().contains(\"too large to read into memory\"))\n}","tryCatchPattern":null,"preventionTips":["Stat file: dependencies for a believable size before install when on FUSE/pseudo filesystems","Keep file: dependency artifacts on ordinary disks"],"tags":["tarball","integer-overflow","local-dependency","rust"],"backgroundTag":"integer-overflow","analyzedSha":"6261b7f388016d57ca6b90340342411cd1d0d00f","analyzedAt":"2026-08-17T18:30:54.750Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}