{"record":{"id":"e10707ead21c2f43","repo":"clockworklabs/SpacetimeDB","slug":"refusing-to-compress-mutable-segment-head-offset","errorCode":null,"errorMessage":"refusing to compress mutable segment {head_offset}","messagePattern":"refusing to compress mutable segment (.+?)","errorType":"exception","errorClass":"std::io::Error","httpStatus":null,"severity":"error","filePath":"crates/commitlog/src/lib.rs","lineNumber":389,"sourceCode":"    ///\n    /// This method acquires a read lock on this `Commitlog` instance, but\n    /// releases it once the compression work starts. Concurrent compression\n    /// tasks on the same segment are safe, but external coordination is\n    /// required to avoid duplicate work.\n    ///\n    /// Attempting to compress a segment that is already compressed incurs a\n    /// small overhead to open the file and determining its format, but\n    /// otherwise does nothing.\n    pub fn compress_segments(&self, offsets: &[u64]) -> io::Result<CompressionStats> {\n        let (repo, head_offset) = {\n            let inner = self.inner.read().unwrap();\n            let repo = inner.repo.clone();\n            let head_offset = inner.head.min_tx_offset();\n\n            (repo, head_offset)\n        };\n        if offsets.contains(&head_offset) {\n            return Err(io::Error::new(\n                io::ErrorKind::InvalidInput,\n                format!(\"refusing to compress mutable segment {head_offset}\"),\n            ));\n        }\n        let mut stats = <_>::default();\n        for offset in offsets {\n            stats += repo.compress_segment(*offset)?;\n        }\n        Ok(stats)\n    }\n\n    /// Remove all data from the log and reopen it.\n    ///\n    /// Log segments are deleted starting from the newest. As multiple segments\n    /// cannot be deleted atomically, the log may not be completely empty if\n    /// the method returns an error.\n    ///\n    /// Note that the method consumes `self` to ensure the log is not modified","sourceCodeStart":371,"sourceCodeEnd":407,"githubUrl":"https://github.com/clockworklabs/SpacetimeDB/blob/6dee26c6efc2856793e12b148a59742964f5d783/crates/commitlog/src/lib.rs#L371-L407","documentation":"Commitlog::compress_segments refuses to compress the segment that currently holds the log head: the head segment is mutable (still receiving commits), and compressing it would mean re-compressing on every append. The check compares the requested offsets against the head's min_tx_offset and returns InvalidInput naming the offending offset when it is included.","triggerScenarios":"Passing the head/mutable segment's offset in the offsets slice - commonly by computing 'all segment offsets' from a directory listing or segment count without excluding the newest, or racing a segment rollover between listing offsets and calling compress_segments.","commonSituations":"Maintenance jobs that compress every segment file found on disk; off-by-one when deriving the last segment offset; compressing concurrently with active writes.","solutions":["Exclude the head segment: obtain the current head offset from the log and filter it out of the list before calling compress_segments.","Compress only sealed segments - any segment strictly older than the head is safe.","If offsets come from a directory scan, re-check the head immediately before the call to avoid rollover races."],"exampleFix":"// before\nlet offsets = list_all_segment_offsets(); // includes the head segment\ncommitlog.compress_segments(&offsets)?;\n\n// after\nlet head = commitlog.head_min_tx_offset();\nlet sealed: Vec<u64> = list_all_segment_offsets().into_iter().filter(|&o| o != head).collect();\ncommitlog.compress_segments(&sealed)?;","handlingStrategy":"validation","validationCode":"// Filter out the mutable head segment before compressing\nlet head = commitlog.head_min_tx_offset();\nlet sealed: Vec<u64> = requested_offsets.into_iter().filter(|&o| o != head).collect();\ncommitlog.compress_segments(&sealed)?;","typeGuard":null,"tryCatchPattern":"match commitlog.compress_segments(&offsets) {\n    Ok(stats) => stats,\n    Err(e) if e.kind() == io::ErrorKind::InvalidInput && e.to_string().contains(\"mutable segment\") => {\n        // head rolled over since the list was built: drop the head offset and retry with the rest\n        let head = commitlog.head_min_tx_offset();\n        let rest: Vec<u64> = offsets.into_iter().filter(|&o| o != head).collect();\n        commitlog.compress_segments(&rest)?\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Define 'compressible' as strictly older than the head segment and encode it in one shared helper.","Source compression targets from the log API, not from filesystem listings.","Re-check the head offset immediately before calling compress_segments to avoid rollover races."],"tags":["rust","commitlog","compression","segments","api-misuse"],"backgroundTag":"compress-active-segment","analyzedSha":"6dee26c6efc2856793e12b148a59742964f5d783","analyzedAt":"2026-08-20T06:08:37.179Z","contentChangedAt":"2026-08-20T06:08:37.179Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}