{"record":{"id":"efa7176311309045","repo":"clockworklabs/SpacetimeDB","slug":"all-zeroes-commit-header","errorCode":null,"errorMessage":"all-zeroes commit header","messagePattern":"all-zeroes commit header","errorType":"exception","errorClass":"io::Error","httpStatus":null,"severity":"error","filePath":"crates/commitlog/src/stream/writer.rs","lineNumber":306,"sourceCode":"                // The stream is exhausted, break the outer loop.\n                trace!(\"eof\");\n                return Ok(AppendInnerResult::StreamExhausted);\n            };\n            if buf.starts_with(&segment::MAGIC) {\n                // New segment, break inner loop.\n                trace!(\"segment marker\");\n                return Ok(AppendInnerResult::SegmentMarker);\n            }\n\n            // Read the header, so we can determine the size of the commit.\n            if read_exact(stream, &mut self.commit_buf.header).await?.is_eof() {\n                return Ok(AppendInnerResult::StreamExhausted);\n            }\n            let Some(commit_header) = commit::Header::decode(&self.commit_buf.header[..])\n                .inspect_err(|e| warn!(\"failed to decode commit header: {e}\"))?\n            else {\n                // Nb. eof handled above.\n                return Err(io::Error::new(io::ErrorKind::InvalidData, \"all-zeroes commit header\"));\n            };\n\n            // Read the rest of the commit.\n            self.commit_buf.body.resize(\n                commit_header.len as usize + CHECKSUM_LEN[current_segment.header.checksum_algorithm as usize],\n                0,\n            );\n            stream.read_exact(&mut self.commit_buf.body).await?;\n            // Decode the commit and verify its checksum.\n            let commit = StoredCommit::decode(self.commit_buf.as_reader())\n                .inspect_err(|e| warn!(\"failed to decode commit: {e}\"))?\n                .expect(\"commit decode cannot return `None` because we already decoded the header\");\n\n            // Check that the commit offset is what we expect.\n            let expected_offset = self\n                .last_written_tx_range\n                .as_ref()\n                .map(|range| range.end)","sourceCodeStart":288,"sourceCodeEnd":324,"githubUrl":"https://github.com/clockworklabs/SpacetimeDB/blob/524b4487d949b61a07d4f39c862d1290259dfd20/crates/commitlog/src/stream/writer.rs#L288-L324","documentation":"While consuming commits from the input stream, EOF is handled explicitly just above, so the next bytes must be a commit header. commit::Header::decode returned None, which per the code happens only for an all-zero header. Zeros at this position mean the stream is misaligned with its framing, or the producer is emitting uninitialized/preallocated bytes.","triggerScenarios":"An earlier record declared a wrong length, shifting subsequent reads into zero padding; streaming a segment file past its last committed write into fallocate-preallocated zero space; a producer bug or truncated/corrupted feed; a checksum-algorithm mismatch changing body sizes and desynchronizing the reader.","commonSituations":"Reading a commitlog segment up to its file size instead of its committed length after a writer crash; nodes on both ends configured with different checksum algorithms; bit rot or truncation in transit.","solutions":["Stream only up to the committed length of each segment, never to the file's allocated size.","Verify both ends run the same commitlog format version and checksum algorithm.","Restart the stream from the last known-good segment boundary or a fresh snapshot."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"fn is_all_zero(buf: &[u8]) -> bool {\n    buf.iter().all(|&b| b == 0)\n}\n\n// Before feeding a raw region to the stream writer, reject all-zero buffers:\n// they indicate padding or unwritten preallocated space, not valid commits.","typeGuard":null,"tryCatchPattern":"match writer.append_all(stream, progress).await {\n    Err(e) if e.kind() == io::ErrorKind::InvalidData && e.to_string().contains(\"all-zeroes commit header\") => {\n        // Framing is misaligned: restart the stream from the last good segment boundary.\n    }\n    r => r,\n}","preventionTips":["Stream only up to each segment's committed length, never its allocated file size.","Keep checksum-algorithm and format-version settings identical on both ends.","Treat any zero-filled tail in a source segment as unwritten and stop there."],"tags":["commitlog","streaming","corruption","rust"],"backgroundTag":"stream-data-corruption","analyzedSha":"524b4487d949b61a07d4f39c862d1290259dfd20","analyzedAt":"2026-08-16T23:58:54.611Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}