EpicGames/lore · error · io::Error

file shrank while reading

Error message

file shrank while reading

What it means

read_file_bytes observed a zero-byte read while filling a buffer of the size taken from the file's own metadata — meaning the file shrank (was truncated or replaced) between the metadata read and the data read. ErrorKind::UnexpectedEof is raised so the caller never receives a short, partially-stale buffer.

Solutions

1. Re-open the file and retry the read_file_bytes call; the new file is likely stable.
2. Read the file once into an owned snapshot (e.g. copy to a temp path or open + fstat the same fd) to avoid the race.
3. Coordinate with the writer: use file locking or read after the writer signals completion.
4. Tolerate short reads by looping with plain positioned reads and accepting fewer bytes.

Example fix

// before
let bytes = psync::read_file_bytes(&options, path)?; // fails if file shrinks mid-read
// after
let bytes = loop {
    match psync::read_file_bytes(&options, path) {
        Ok(b) => break b,
        Err(e) if e.kind() == std::io::ErrorKind::UnexpectedEof => continue, // retried on rotation
        Err(e) => return Err(e),
    }
};

// Rust
let before = fs::metadata(path)?.len();
// after read_file_bytes succeeds, confirm the file did not change:
if fs::metadata(path)?.len() != before { /* retry */ }

// Rust
let bytes = loop {
    match psync::read_file_bytes(&options, path) {
        Ok(b) => break Ok(b),
        Err(e) if e.kind() == std::io::ErrorKind::UnexpectedEof && attempts < 3 => { attempts += 1; }
        Err(e) => break Err(e),
    }
}?;

When it happens

Trigger: Calling read_file_bytes on a file that another process truncates or atomically replaces (rename-over) between the stat and the read loop; reading a log/tmp file that gets rotated during the read.

Common situations: Log rotation truncating the file being read; a build/cache process rewriting the file in place; reading /proc-style or temp files that shrink; TOCTOU race in a monitoring tool.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• file ended before the requested read length
• file refused further writes
• file refused further writes
• file ended before the requested read length
• not a regular file

AI-assisted analysis of EpicGames/lore@074eb0b0d1 (2026-09-13). Data as JSON: /api/errors/dbc3f57147cd5e5f. Report an issue: GitHub.

Appendix: source

Thrown at lore-io/src/psync.rs:155

    }

    pub(crate) async fn read_file_bytes(&self, path: PathBuf) -> std::io::Result<Bytes> {
        SyscallPool::global()
            .submit(move || {
                let file = crate::file::OpenOptions::new()
                    .read(true)
                    .to_std_blocking()
                    .open(path)?;
                let len = file.metadata()?.len() as usize;
                crate::driver::check_whole_file_len(len)?;
                // SAFETY: every byte up to `len` is filled before returning, and a file that
                // shrank returns an error rather than the buffer.
                let mut buffer = unsafe { crate::buffer::uninit_buffer(len) };
                let mut done = 0;
                while done < len {
                    let read = read_at_impl(&file, &mut buffer[done..len], done as u64)?;
                    if read == 0 {
                        return Err(std::io::Error::new(
                            std::io::ErrorKind::UnexpectedEof,
                            "file shrank while reading",
                        ));
                    }
                    done += read;
                }
                Ok(buffer.freeze())
            })
            .await
    }

    pub(crate) async fn write_file_bytes(
        &self,
        path: PathBuf,
        data: Bytes,
        durable: bool,
    ) -> std::io::Result<std::fs::Metadata> {
        SyscallPool::global()

View on GitHub (pinned to 074eb0b0d1)