{"record":{"id":"3dfa7045b9403527","repo":"clockworklabs/SpacetimeDB","slug":"unexpected-eof-while-validating-commit-at-byte-off","errorCode":null,"errorMessage":"unexpected EOF while validating commit at byte offset {byte_offset}","messagePattern":"unexpected EOF while validating commit at byte offset (.+?)","errorType":"exception","errorClass":"io::Error","httpStatus":null,"severity":"error","filePath":"crates/commitlog/src/segment.rs","lineNumber":531,"sourceCode":"\n/// Try to extract the commit header from the asked position without advancing seek.\n/// `IndexFileMut` fsync asynchoronously, which makes it important for reader to verify its entry\nfn validate_commit_at_byte_offset<Reader: io::Read + io::Seek>(\n    mut reader: &mut Reader,\n    byte_offset: u64,\n) -> io::Result<commit::Header> {\n    let pos = reader.stream_position()?;\n    reader.seek(SeekFrom::Start(byte_offset))?;\n\n    let hdr_or_error = StoredCommit::decode(&mut reader).and_then(|maybe_commit| {\n        let StoredCommit {\n            min_tx_offset,\n            epoch,\n            n,\n            records,\n            ..\n        } = maybe_commit.ok_or_else(|| {\n            io::Error::new(\n                ErrorKind::UnexpectedEof,\n                format!(\"unexpected EOF while validating commit at byte offset {byte_offset}\"),\n            )\n        })?;\n\n        Ok(commit::Header {\n            min_tx_offset,\n            epoch,\n            n,\n            len: records.len() as u32,\n        })\n    });\n\n    // Restore the original position\n    reader.seek(SeekFrom::Start(pos))?;\n\n    hdr_or_error\n}","sourceCodeStart":513,"sourceCodeEnd":549,"githubUrl":"https://github.com/clockworklabs/SpacetimeDB/blob/524b4487d949b61a07d4f39c862d1290259dfd20/crates/commitlog/src/segment.rs#L513-L549","documentation":"validate_commit_at_byte_offset tried to decode a commit at the byte offset supplied by the index and hit end-of-stream before a complete commit header could be read (UnexpectedEof). The index points past the data end or into the middle of the last, incomplete commit - the classic torn-write signature, possible when index entries become visible before the data is durable.","triggerScenarios":"Process killed mid-commit while offset_index_require_segment_fsync = false (index updated before data was fsynced); a segment truncated while its index retained the old entry; partial page writes on power loss.","commonSituations":"Crash and fuzz testing; hard power-off on local disks; directories touched by external tooling while the log is in use.","solutions":["Delete the segment's .idx so it is rebuilt from the durable data","Keep offset_index_require_segment_fsync = true (default) to close the durability window","If it recurs with fsync enabled, suspect the filesystem/device of losing writes and investigate"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":"fn is_index_points_past_eof(e: &io::Error) -> bool {\n    e.kind() == io::ErrorKind::UnexpectedEof\n        && e.to_string().contains(\"validating commit at byte offset\")\n}","tryCatchPattern":"match reader.seek_to_offset(&index, offset) {\n    Ok(_) => { /* proceed */ }\n    Err(e) if is_index_points_past_eof(&e) => {\n        // stale index entry after a torn write: rebuild index, rescan from segment start\n        drop_the_index_file_for_this_segment()?;\n        reopen_and_rescan_from_segment_start(offset)?;\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Leave offset_index_require_segment_fsync at its default true to close the torn-write window","Run crash/kill tests that cover the moment between data write and index flush","If the error appears with fsync on, audit the storage stack for lost writes"],"tags":["rust","commitlog","unexpected-eof","partial-write","offset-index","crash-recovery"],"backgroundTag":"partial-write","analyzedSha":"524b4487d949b61a07d4f39c862d1290259dfd20","analyzedAt":"2026-08-16T23:58:54.611Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}