clockworklabs/SpacetimeDB · error · std::io::Error

refusing to compress mutable segment {head_offset}

Error message

refusing to compress mutable segment {head_offset}

What it means

Commitlog::compress_segments refuses to compress the segment that currently holds the log head: the head segment is mutable (still receiving commits), and compressing it would mean re-compressing on every append. The check compares the requested offsets against the head's min_tx_offset and returns InvalidInput naming the offending offset when it is included.

Source

Thrown at crates/commitlog/src/lib.rs:389

    ///
    /// This method acquires a read lock on this `Commitlog` instance, but
    /// releases it once the compression work starts. Concurrent compression
    /// tasks on the same segment are safe, but external coordination is
    /// required to avoid duplicate work.
    ///
    /// Attempting to compress a segment that is already compressed incurs a
    /// small overhead to open the file and determining its format, but
    /// otherwise does nothing.
    pub fn compress_segments(&self, offsets: &[u64]) -> io::Result<CompressionStats> {
        let (repo, head_offset) = {
            let inner = self.inner.read().unwrap();
            let repo = inner.repo.clone();
            let head_offset = inner.head.min_tx_offset();

            (repo, head_offset)
        };
        if offsets.contains(&head_offset) {
            return Err(io::Error::new(
                io::ErrorKind::InvalidInput,
                format!("refusing to compress mutable segment {head_offset}"),
            ));
        }
        let mut stats = <_>::default();
        for offset in offsets {
            stats += repo.compress_segment(*offset)?;
        }
        Ok(stats)
    }

    /// Remove all data from the log and reopen it.
    ///
    /// Log segments are deleted starting from the newest. As multiple segments
    /// cannot be deleted atomically, the log may not be completely empty if
    /// the method returns an error.
    ///
    /// Note that the method consumes `self` to ensure the log is not modified

View on GitHub (pinned to 6dee26c6ef)

Solutions

1. Exclude the head segment: obtain the current head offset from the log and filter it out of the list before calling compress_segments.
2. Compress only sealed segments - any segment strictly older than the head is safe.
3. If offsets come from a directory scan, re-check the head immediately before the call to avoid rollover races.

Example fix

// before
let offsets = list_all_segment_offsets(); // includes the head segment
commitlog.compress_segments(&offsets)?;

// after
let head = commitlog.head_min_tx_offset();
let sealed: Vec<u64> = list_all_segment_offsets().into_iter().filter(|&o| o != head).collect();
commitlog.compress_segments(&sealed)?;

// Filter out the mutable head segment before compressing
let head = commitlog.head_min_tx_offset();
let sealed: Vec<u64> = requested_offsets.into_iter().filter(|&o| o != head).collect();
commitlog.compress_segments(&sealed)?;

match commitlog.compress_segments(&offsets) {
    Ok(stats) => stats,
    Err(e) if e.kind() == io::ErrorKind::InvalidInput && e.to_string().contains("mutable segment") => {
        // head rolled over since the list was built: drop the head offset and retry with the rest
        let head = commitlog.head_min_tx_offset();
        let rest: Vec<u64> = offsets.into_iter().filter(|&o| o != head).collect();
        commitlog.compress_segments(&rest)?
    }
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Passing the head/mutable segment's offset in the offsets slice - commonly by computing 'all segment offsets' from a directory listing or segment count without excluding the newest, or racing a segment rollover between listing offsets and calling compress_segments.

Common situations: Maintenance jobs that compress every segment file found on disk; off-by-one when deriving the last segment offset; compressing concurrently with active writes.

Related errors

• segment {offset} already exists
• segment {offset} does not exist
• repo {}: too many empty segments: {}
• repo {}: segment {} already exists and is non-empty
• failed to read {} bytes of commit payload: {}

AI-assisted analysis of clockworklabs/SpacetimeDB@6dee26c6ef (2026-08-20). Data as JSON: /api/errors/e10707ead21c2f43. Report an issue: GitHub.