clockworklabs/SpacetimeDB · error · io::Error

failed to read segment header ({} bytes): {}

Error message

failed to read segment header ({} bytes): {}

What it means

Header::decode could not read the fixed 10-byte segment header: read_exact failed and the original ErrorKind and message pass through - typically UnexpectedEof when the file is shorter than 10 bytes, otherwise a genuine I/O error. It fires when a segment file exists but is truncated, empty, or partially written.

Source

Thrown at crates/commitlog/src/segment.rs:50

pub struct Header {
    pub log_format_version: u8,
    pub checksum_algorithm: u8,
}

impl Header {
    pub const LEN: usize = MAGIC.len() + /* log_format_version + checksum_algorithm + reserved + reserved */ 4;

    pub fn write<W: io::Write>(&self, mut out: W) -> io::Result<()> {
        out.write_all(&MAGIC)?;
        out.write_all(&[self.log_format_version, self.checksum_algorithm, 0, 0])?;

        Ok(())
    }

    pub fn decode<R: io::Read>(mut read: R) -> io::Result<Self> {
        let mut buf = [0; Self::LEN];
        read.read_exact(&mut buf).map_err(|e| {
            io::Error::new(
                e.kind(),
                format!("failed to read segment header ({} bytes): {}", Self::LEN, e),
            )
        })?;

        if !buf.starts_with(&MAGIC) {
            return Err(io::Error::new(
                io::ErrorKind::InvalidData,
                format!(
                    "segment header does not start with magic: expected {:02x?}, got {:02x?}",
                    MAGIC,
                    &buf[..MAGIC.len()]
                ),
            ));
        }

        Ok(Self {
            log_format_version: buf[MAGIC.len()],

View on GitHub (pinned to 524b4487d9)

Solutions

1. Stat the file: if its size is under 10 bytes it is a crash remnant holding no commits - quarantine/delete it and reopen
2. If the size looks right, chase the inner I/O error (device health, permissions)
3. Restore the segment from backup if it was expected to contain committed data

// reject sub-header files before trying to open them as segments
const SEGMENT_HEADER_LEN: u64 = 10; // segment::Header::LEN
for entry in std::fs::read_dir(dir)? {
    let p = entry?.path();
    if p.extension().is_none_or(|e| e != "idx" && e != "lock") {
        if let Ok(md) = std::fs::metadata(&p) {
            if md.len() < SEGMENT_HEADER_LEN {
                return Err(io::Error::new(io::ErrorKind::InvalidData, format!("{}: too short to be a segment", p.display())));
            }
        }
    }
}

fn is_header_read_failure(e: &io::Error) -> bool {
    e.to_string().starts_with("failed to read segment header")
}

match open_segment(&path) {
    Ok(seg) => seg,
    Err(e) if is_header_read_failure(&e) && e.kind() == io::ErrorKind::UnexpectedEof => {
        // crash remnant shorter than a header: quarantine and continue
        std::fs::rename(&path, path.with_extension("quarantine"))?;
        open_segment(&path)?
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: Opening a segment file of 0-9 bytes (crash between file creation and header write, or truncation); an I/O error from the device while reading the header; a foreign/renamed file sitting where a segment is expected.

Common situations: Power loss during segment creation; restoring from an incomplete backup; files truncated after a disk-full event.

Related errors

• error reading commit header: {e}
• failed to read {} bytes of commit payload: {}
• failed to read checksum: {e}
• segment header does not start with magic: expected {:02x?},
• unexpected EOF while validating commit at byte offset {byte_

AI-assisted analysis of clockworklabs/SpacetimeDB@524b4487d9 (2026-08-16). Data as JSON: /api/errors/a084b209e9b14576. Report an issue: GitHub.