clockworklabs/SpacetimeDB · critical · io::Error

InvalidData

InvalidData

Error message

no valid commits in segment

What it means

During log open/resume (resume_segment_writer), a segment file longer than its 10-byte header was traversed but its Metadata ended with an empty transaction range: not a single complete commit could be decoded. The library refuses to resume such a segment (InvalidData) because it cannot distinguish a crash remnant (header + torn first commit) from real corruption.

Source

Thrown at crates/commitlog/src/repo/mod.rs:323

        .open_segment_reader(offset)
        .map_err(|source| with_segment_context("opening segment for resume", repo, offset, source))?;

    // If the segment at `offset` is empty, remove it and try the previous.
    // Return an error if no previous segment is found.
    let len = reader
        .segment_len()
        .map_err(|source| with_segment_context("determining segment file size for resume", repo, offset, source))?;
    if len <= segment::Header::LEN as u64 {
        debug!("repo {}: segment {} is empty", repo, offset);
        return Ok(ResumedSegment::Empty);
    }

    let guard_non_empty = |meta: &Metadata| match meta.tx_range.is_empty() {
        true => Err(with_segment_context(
            "checking metadata",
            repo,
            offset,
            io::Error::new(io::ErrorKind::InvalidData, "no valid commits in segment"),
        )),
        false => Ok(()),
    };

    // The segment is now guaranteed to be non-empty, i.e. contain more bytes
    // than the segment header.
    //
    // Traverse it to gather the `Metadata` and ensure that the segment is safe
    // to resume, which is the case if:
    //
    // - it contains at least one commit
    // - it does not contain corrupted commits
    // - the existing segment passes the compatibility check
    // - the existing segment's version is the same as
    //   the one requested in `opts`
    let offset_index = repo.get_offset_index(offset).ok();
    let meta = match Metadata::extract(offset, &mut reader, offset_index.as_ref()) {
        Err(error::SegmentMetadata::InvalidCommit { sofar, source }) => {

View on GitHub (pinned to 524b4487d9)

Solutions

1. The segment contains zero valid commits, so nothing committed is lost: quarantine/delete the named segment file and reopen - the log recreates it
2. If anything downstream might have depended on that segment, restore it from a backup/replica instead of deleting
3. If it reproduces without any preceding crash, preserve the file and report it as a bug

fn is_no_valid_commits(e: &io::Error) -> bool {
    e.kind() == io::ErrorKind::InvalidData
        && e.to_string().contains("no valid commits in segment")
}

match Commitlog::open(dir, opts, None) {
    Ok(log) => log,
    Err(e) if is_no_valid_commits(&e) => {
        // segment holds zero valid commits: quarantine it and retry open once
        let path = extract_segment_path(&e); // path precedes the bracketed context
        std::fs::rename(&path, path.with_extension("quarantine"))?;
        Commitlog::open(dir, opts, None)?
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: A crash after the segment header was written but before the first commit completed, leaving only partial commit bytes; truncation or bit-rot destroying the only commit in a segment; a foreign file with a valid header but garbage body.

Common situations: Power loss right after a segment roll; kill -9 during the very first commit; partially restored data directories.

Related errors

• out-of-order offset: expected={} actual={}
• failed to decode commit
• mismatch key in index offset file: expected={} actual={}
• failed to read {} bytes of commit payload: {}
• failed to read checksum: {e}

AI-assisted analysis of clockworklabs/SpacetimeDB@524b4487d9 (2026-08-16). Data as JSON: /api/errors/7ceaa0db0d22ac83. Report an issue: GitHub.