clockworklabs/SpacetimeDB · error · io::Error

segment header does not start with magic: expected

Error message

segment header does not start with magic: expected {:02x?}, got {:02x?}

What it means

After successfully reading 10 bytes, Header::decode requires the first 6 bytes to equal the MAGIC constant b"(ds)^2". A different prefix returns io::ErrorKind::InvalidData with expected/got hex dumps. The file exists and is long enough, but it is not a commitlog segment.

Solutions

1. Inspect the first bytes (xxd file | head -1) and compare with 28 64 73 29 5e 32 ('(ds)^2')
2. Point the repo at the correct data directory containing only commitlog segments
3. Quarantine or delete the stray file after confirming it is not a misnamed segment
4. Restore the segment from a replica or backup if it is genuinely corrupted

const MAGIC: &[u8; 6] = b"(ds)^2";
fn is_segment_file(path: &std::path::Path) -> io::Result<bool> {
    use std::io::Read as _;
    let mut f = std::fs::File::open(path)?;
    let mut magic = [0u8; 6];
    match f.read_exact(&mut magic) {
        Ok(()) => Ok(&magic == MAGIC),
        Err(e) if e.kind() == io::ErrorKind::UnexpectedEof => Ok(false),
        Err(e) => Err(e),
    }
}

fn is_bad_magic(err: &io::Error) -> bool {
    err.kind() == io::ErrorKind::InvalidData && err.to_string().contains("does not start with magic")
}

match segment::Header::decode(&mut file) {
    Ok(h) => Ok(h),
    Err(e) if is_bad_magic(&e) => {
        // do not delete blindly: quarantine and alert; likely wrong directory
        quarantine(&path)?;
        Err(e)
    }
    Err(e) => Err(e),
}

When it happens

Trigger: Pointing the repo at a directory containing unrelated files that get picked up as segments; a segment file overwritten or corrupted so the header is gone; reading a file produced by a different product or an incompatible fork.

Common situations: Misconfigured repo path (e.g. data dir shared with other storage); stray files dropped into the log directory; corruption at offset 0; restoring wrong files from backup.

Related errors

• no valid commits in segment
• out-of-order offset: expected=
• all-zeroes commit header
• Delete for non-existent row when replaying transaction
• Error looking up name for truncated table

AI-assisted analysis of clockworklabs/SpacetimeDB@3653d2ed49 (2026-08-20). Data as JSON: /api/errors/1d8e49bb8d23d59f. Report an issue: GitHub.

Appendix: source

Thrown at crates/commitlog/src/segment.rs:57

    pub fn write<W: io::Write>(&self, mut out: W) -> io::Result<()> {
        out.write_all(&MAGIC)?;
        out.write_all(&[self.log_format_version, self.checksum_algorithm, 0, 0])?;

        Ok(())
    }

    pub fn decode<R: io::Read>(mut read: R) -> io::Result<Self> {
        let mut buf = [0; Self::LEN];
        read.read_exact(&mut buf).map_err(|e| {
            io::Error::new(
                e.kind(),
                format!("failed to read segment header ({} bytes): {}", Self::LEN, e),
            )
        })?;

        if !buf.starts_with(&MAGIC) {
            return Err(io::Error::new(
                io::ErrorKind::InvalidData,
                format!(
                    "segment header does not start with magic: expected {:02x?}, got {:02x?}",
                    MAGIC,
                    &buf[..MAGIC.len()]
                ),
            ));
        }

        Ok(Self {
            log_format_version: buf[MAGIC.len()],
            checksum_algorithm: buf[MAGIC.len() + 1],
        })
    }

    pub fn ensure_compatible(&self, max_log_format_version: u8, checksum_algorithm: u8) -> Result<(), String> {
        if self.log_format_version > max_log_format_version {
            return Err(format!("unsupported log format version: {}", self.log_format_version));

View on GitHub (pinned to 3653d2ed49)