quickwit-oss/tantivy · error · io::Error
UnexpectedEof
UnexpectedEof
Error message
failed to fill whole buffer
What it means
OwnedBytes implements std::io::ReadExact; this error is thrown when read_exact was asked to fill a buffer of N bytes but the underlying bytes/mmap only supplied fewer. It is tantivy's typed UnexpectedEof for short reads on in-memory owned byte slices.
Source
Thrown at ownedbytes/src/lib.rs:233
Ok(buf_len)
} else {
buf[..data_len].copy_from_slice(self.data);
self.data = &[];
Ok(data_len)
}
}
#[inline]
fn read_to_end(&mut self, buf: &mut Vec<u8>) -> io::Result<usize> {
buf.extend(self.data);
let read_len = self.data.len();
self.data = &[];
Ok(read_len)
}
#[inline]
fn read_exact(&mut self, buf: &mut [u8]) -> io::Result<()> {
let read_len = self.read(buf)?;
if read_len != buf.len() {
return Err(io::Error::new(
io::ErrorKind::UnexpectedEof,
"failed to fill whole buffer",
));
}
Ok(())
}
}
#[cfg(test)]
mod tests {
use std::io::{self, Read};
use super::OwnedBytes;
#[test]
fn test_owned_bytes_debug() {
let short_bytes = OwnedBytes::new(b"abcd".as_ref());
assert_eq!(View on GitHub (pinned to b5d8deb80c)
Solutions
- Check the byte-slice length before calling read_exact and size the buffer to the available data
- Regenerate or re-fetch the truncated/corrupted file
- In tests, make the fixture buffer at least as large as what read_exact requests
- Review range/slice computations (slice offsets, header length constants) for off-by-one errors
Example fix
// before
let mut buf = [0u8; 8];
bytes.read_exact(&mut buf)?; // panics/errors if bytes.len() < 8
// after
let mut buf = [0u8; 8];
if bytes.len() < 8 {
return Err(io::Error::new(io::ErrorKind::UnexpectedEof, "need 8 bytes"));
}
bytes.read_exact(&mut buf)?; Defensive patterns
Strategy: validation
Validate before calling
if bytes.len() < needed { return Err(io::Error::new(io::ErrorKind::UnexpectedEof, "too short")); } Try / catch
match read_exact_result {
Err(e) if e.kind() == io::ErrorKind::UnexpectedEof => {
eprintln!("short read: need {} bytes", buf.len());
}
Err(e) => return Err(e),
Ok(()) => (),
} Prevention
- Size buffers from the source length, not hardcoded constants
- Re-verify downloaded/copied index files (checksums, size checks)
- Double-check slice ranges when building OwnedBytes::slice calls
- In tests, build fixtures with actual serialization output
When it happens
Trigger: read_exact called on an OwnedBytes whose length is smaller than the requested buffer size: deserializing a struct from a byte slice shorter than needed, slicing a file with an out-of-range range (e.g. bytes.slice(0, huge_len)), or reading past the end of a mmap-backed segment.
Common situations: Hand-written deserialization tests (as in test_owned_bytes_read) requesting more bytes than the fixture contains; corrupted/truncated segment files whose header advertises more data than exists; off-by-one in slice arithmetic when parsing custom file formats.
Related errors
AI-assisted analysis of quickwit-oss/tantivy@b5d8deb80c (2026-09-05).
Data as JSON: /api/errors/69d45983a4beb830.
Report an issue: GitHub.