databendlabs/databend · error · UnexpectedEof

Invalid cache item, expects

Error message

Invalid cache item, expects {} bytes, got {}, path {:?}

What it means

read_cache_content_with_syscall reads a disk-cache entry and expects exactly `size` bytes; a read returning 0 (EOF before the expected length) means the cache file is truncated or corrupted, so it raises an UnexpectedEof io error describing expected vs got bytes and the cache path.

Solutions

1. Delete the affected cache file (path is in the error) or clear the whole disk cache directory and let it repopulate.
2. Check disk space and filesystem health (dmesg, fsck) for the cache volume.
3. Upgrade/restart the query node if a stale index maps to the truncated file; disable disk cache temporarily if corruption recurs.

Example fix

// before (keep serving from corrupt cache)
// after
$ rm /var/lib/databend/cache/<path-from-error>
# then restart the query node or let the cache re-download the block

let meta = std::fs::metadata(&path)?;
if (meta.len() as u64) < expected_size { bail!("cache file truncated: {:?}", path); }

fn cache_file_intact(path: &Path, expected: u64) -> bool {
    std::fs::metadata(path).map(|m| m.len() >= expected).unwrap_or(false)
}

match read_result {
    Err(e) if e.kind() == std::io::ErrorKind::UnexpectedEof => {
        // evict the corrupt entry (path is embedded in the message) and retry the read
        invalidate_cache_entry(&path).await?;
        retry().await
    }
    other => other,
}

When it happens

Trigger: Reading a block from the disk cache (LRU-indexed) where the underlying cache file is shorter than the recorded size — partial write, crash during write, manual deletion/truncation, or stale LRU index pointing at a rewritten file.

Common situations: Node crash/power loss during cache writes; disk full causing truncated writes; sharing a cache directory between versions; corrupted cache volume.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Expected to have bytes
• Expected to have insensitive bytes
• Expected to ignore a byte
• Fst data is corrupted
• Must reach the buffer end

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/1dad4ff4e6598da3. Report an issue: GitHub.

Appendix: source

Thrown at src/query/storages/common/cache/src/providers/disk_cache/disk_cache_lru.rs:265

        let mut vec = Vec::<u8>::with_capacity(size);
        let vec_ptr = vec.as_mut_ptr();
        let fd = file.as_raw_fd();
        let mut total_read = 0;

        while total_read < size {
            let result = unsafe {
                read_syscall.read(
                    fd,
                    vec_ptr.add(total_read) as *mut libc::c_void,
                    (size - total_read) as libc::size_t,
                )
            };

            match result {
                n if n > 0 => total_read += n as usize,
                0 => {
                    return Err(std::io::Error::new(
                        std::io::ErrorKind::UnexpectedEof,
                        format!(
                            "Invalid cache item, expects {} bytes, got {}, path {:?}",
                            size, total_read, path
                        ),
                    ));
                }
                _ => {
                    let err = std::io::Error::last_os_error();
                    if err.raw_os_error() == Some(libc::EINTR) {
                        continue;
                    } else {
                        return Err(err);
                    }
                }
            }
        }

View on GitHub (pinned to 288d84d76e)