{"record":{"id":"51c2caf395fa9bf2","repo":"clockworklabs/SpacetimeDB","slug":"expected-commit-offset-but-encountered","errorCode":null,"errorMessage":"expected commit offset {} but encountered {}","messagePattern":"expected commit offset (.+?) but encountered (.+?)","errorType":"exception","errorClass":"io::Error","httpStatus":null,"severity":"error","filePath":"crates/commitlog/src/stream/writer.rs","lineNumber":327,"sourceCode":"            // Read the rest of the commit.\n            self.commit_buf.body.resize(\n                commit_header.len as usize + CHECKSUM_LEN[current_segment.header.checksum_algorithm as usize],\n                0,\n            );\n            stream.read_exact(&mut self.commit_buf.body).await?;\n            // Decode the commit and verify its checksum.\n            let commit = StoredCommit::decode(self.commit_buf.as_reader())\n                .inspect_err(|e| warn!(\"failed to decode commit: {e}\"))?\n                .expect(\"commit decode cannot return `None` because we already decoded the header\");\n\n            // Check that the commit offset is what we expect.\n            let expected_offset = self\n                .last_written_tx_range\n                .as_ref()\n                .map(|range| range.end)\n                .unwrap_or_default();\n            if commit.min_tx_offset != expected_offset {\n                return Err(io::Error::new(\n                    io::ErrorKind::InvalidData,\n                    format!(\n                        \"expected commit offset {} but encountered {}\",\n                        expected_offset, commit.min_tx_offset\n                    ),\n                ));\n            }\n            trace!(\"received commit {commit:?}\");\n\n            // Write the commit and report progress.\n            current_segment\n                .segment\n                .write_all_buf(&mut self.commit_buf.as_buf())\n                .await?;\n            let written_range = commit.min_tx_offset..(commit.min_tx_offset + commit.n as u64);\n            self.last_written_tx_range = Some(written_range.clone());\n            progress.range_written(written_range);\n","sourceCodeStart":309,"sourceCodeEnd":345,"githubUrl":"https://github.com/clockworklabs/SpacetimeDB/blob/524b4487d949b61a07d4f39c862d1290259dfd20/crates/commitlog/src/stream/writer.rs#L309-L345","documentation":"Contiguity check in append_all_inner: the writer tracks the end of the last written transaction range and requires each incoming commit's min_tx_offset to equal it exactly. A different value means the stream skipped transactions (gap) or replayed already-written ones (overlap), either of which would break the local log's contiguous offset sequence.","triggerScenarios":"The producer drops or filters commits between the requested offset and what it actually sends; resuming a stream from an offset that does not match the writer's last_written_tx_range.end; leader-side retention/compaction having removed commits the follower still needs.","commonSituations":"Follower restart with a stale or hand-picked resume offset; misconfigured filtering on the replication feed; leader retention policy deleting old segments too aggressively.","solutions":["Restart the stream from exactly the end of the last written tx range (query repo/writer metadata for the resume point).","If local state is disposable, wipe the commitlog and re-bootstrap from offset 0 or a snapshot.","Disable any commit filtering between producer and consumer; review leader retention settings."],"exampleFix":"// before: guessing a resume offset\nlet start = requested_tx_offset; // may not match what was already written\n\n// after: resume from what the local log actually contains\nlet meta = repo.metadata()?; // last committed tx range\nlet start = meta.map(|m| m.tx_range.end).unwrap_or(0);","handlingStrategy":"validation","validationCode":"// Before opening the stream, compute the exact resume point from local state:\nlet start_tx = repo.metadata() // or writer-provided last written range\n    .map(|m| m.tx_range.end)\n    .unwrap_or(0);\n// Ask the producer for commits starting exactly at `start_tx`.","typeGuard":null,"tryCatchPattern":"match writer.append_all(stream, progress).await {\n    Err(e) if e.kind() == io::ErrorKind::InvalidData && e.to_string().contains(\"expected commit offset\") => {\n        // Gap or overlap detected: re-bootstrap from a snapshot or restart the stream\n        // at the offset reported in the error message.\n    }\n    r => r,\n}","preventionTips":["Always derive resume offsets from the local log's last written tx range, never hard-code them.","Disable commit filtering between replication peers.","Watch leader retention settings so needed segments are not deleted before followers consume them."],"tags":["commitlog","streaming","offset-gap","rust"],"backgroundTag":"offset-gap-detected","analyzedSha":"524b4487d949b61a07d4f39c862d1290259dfd20","analyzedAt":"2026-08-16T23:58:54.611Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}