clockworklabs/SpacetimeDB · error · io::Error

segment header does not start with magic: expected {:02x?},

Error message

segment header does not start with magic: expected {:02x?}, got {:02x?}

What it means

The first 6 bytes of the file being opened do not equal the segment magic constant (ds)^2 (hex 28 64 73 29 5e 32), so the file is not a commitlog segment (InvalidData). Expect this from a wrong file, foreign content, or a corrupted header.

Source

Thrown at crates/commitlog/src/segment.rs:57

    pub fn write<W: io::Write>(&self, mut out: W) -> io::Result<()> {
        out.write_all(&MAGIC)?;
        out.write_all(&[self.log_format_version, self.checksum_algorithm, 0, 0])?;

        Ok(())
    }

    pub fn decode<R: io::Read>(mut read: R) -> io::Result<Self> {
        let mut buf = [0; Self::LEN];
        read.read_exact(&mut buf).map_err(|e| {
            io::Error::new(
                e.kind(),
                format!("failed to read segment header ({} bytes): {}", Self::LEN, e),
            )
        })?;

        if !buf.starts_with(&MAGIC) {
            return Err(io::Error::new(
                io::ErrorKind::InvalidData,
                format!(
                    "segment header does not start with magic: expected {:02x?}, got {:02x?}",
                    MAGIC,
                    &buf[..MAGIC.len()]
                ),
            ));
        }

        Ok(Self {
            log_format_version: buf[MAGIC.len()],
            checksum_algorithm: buf[MAGIC.len() + 1],
        })
    }

    pub fn ensure_compatible(&self, max_log_format_version: u8, checksum_algorithm: u8) -> Result<(), String> {
        if self.log_format_version > max_log_format_version {
            return Err(format!("unsupported log format version: {}", self.log_format_version));

View on GitHub (pinned to 524b4487d9)

Solutions

1. List the directory and check which files are not real segments (magic check) - usually a path misconfiguration
2. Remove or quarantine the foreign file after confirming nothing else owns it
3. Restore genuinely corrupt segments from backup

fn is_segment_file(p: &std::path::Path) -> io::Result<bool> {
    use std::io::Read as _;
    let mut f = std::fs::File::open(p)?;
    let mut magic = [0u8; 6];
    match f.read_exact(&mut magic) {
        Ok(()) => Ok(magic == *b"(ds)^2"),
        Err(e) if e.kind() == io::ErrorKind::UnexpectedEof => Ok(false),
        Err(e) => Err(e),
    }
}

fn is_bad_magic(e: &io::Error) -> bool {
    e.kind() == io::ErrorKind::InvalidData
        && e.to_string().contains("does not start with magic")
}

When it happens

Trigger: Pointing CommitLogDir at a directory containing unrelated files whose names parse as segment offsets; a segment file overwritten or truncated by another program; garbage written over the header by disk corruption.

Common situations: Sharing a data directory with other tools; misconfigured paths (e.g. the WAL dir of another database); bit rot on old segments.

Related errors

• InvalidData
• failed to read segment header ({} bytes): {}
• mismatched key in offset index file
• out-of-order offset: expected={} actual={}
• No valid commit found in index up to key: {candidate_last_ke

AI-assisted analysis of clockworklabs/SpacetimeDB@524b4487d9 (2026-08-16). Data as JSON: /api/errors/1d8e49bb8d23d59f. Report an issue: GitHub.