clockworklabs/SpacetimeDB · error · io::Error

mismatch key in index offset file: expected={} actual={}

Error message

mismatch key in index offset file: expected={} actual={}

What it means

Same validation pass in validate_commit_at_offset: a commit decoded successfully at the index-recorded byte offset, but its first transaction offset (commit.tx_range.start) differs from the tx_offset key the index claims lives there. The offset index and the segment file disagree about which transaction is stored at that position - a stale or out-of-sync index.

Source

Thrown at crates/commitlog/src/segment.rs:766

        ))
    }

    /// Validates and decodes a commit at `byte_offset` in the segment.
    ///
    /// # Returns
    /// * `Ok(commit::Metadata)` - If a valid commit is found with matching transaction offset
    /// * `Err` - If commit can't be decoded or has mismatched transaction offset
    fn validate_commit_at_offset<R: io::Read + io::Seek>(
        reader: &mut R,
        tx_offset: TxOffset,
        byte_offset: u64,
    ) -> io::Result<commit::Metadata> {
        reader.seek(SeekFrom::Start(byte_offset))?;
        let commit = commit::Metadata::extract(reader)?
            .ok_or_else(|| io::Error::new(ErrorKind::InvalidData, "failed to decode commit"))?;

        if commit.tx_range.start != tx_offset {
            return Err(io::Error::new(
                ErrorKind::InvalidData,
                format!(
                    "mismatch key in index offset file: expected={} actual={}",
                    tx_offset, commit.tx_range.start
                ),
            ));
        }

        Ok(commit)
    }
}

#[cfg(test)]
mod tests {
    use itertools::Itertools;
    use pretty_assertions::assert_matches;
    use spacetimedb_paths::server::CommitLogDir;
    use tempfile::tempdir;

View on GitHub (pinned to 524b4487d9)

Solutions

1. Delete the stale offset index for the offending segment so it is regenerated from segment contents.
2. If both segment and index are suspect, remove the pair and re-replicate that range from a leader or snapshot.
3. Restore the commitlog directory as a whole from a consistent backup.
4. Ensure only one process ever writes a given commitlog repo.

// Before repair/verify, scan the segment sequentially and rebuild the expected
// (tx_offset -> byte_offset) map; compare against the on-disk index and rebuild it
// on any divergence rather than trusting stale entries.

match validate_result {
    Err(e) if e.kind() == io::ErrorKind::InvalidData && e.to_string().contains("mismatch key in index offset file") => {
        // Stale index: delete and regenerate the offset index, then retry verification.
    }
    other => other,
}

When it happens

Trigger: Index entries surviving from a previous incarnation of a segment with the same offset (segment recreated after deletion while the old index was kept); a crash between segment rotation and index flush; manually editing, truncating, or copying index files independently of their segment.

Common situations: Re-bootstrapping a follower into a dirty data directory; restoring segment files from backup but not the indexes (or vice versa); two writers having used the same commitlog repo at different times.

Related errors

• InvalidData
• out-of-order offset: expected={} actual={}
• failed to decode commit
• InvalidInput
• AlreadyExists

AI-assisted analysis of clockworklabs/SpacetimeDB@524b4487d9 (2026-08-16). Data as JSON: /api/errors/8f92bf7b0c744d6a. Report an issue: GitHub.