clockworklabs/SpacetimeDB · error · io::Error

no current segment, expected segment header

Error message

no current segment, expected segment header

What it means

StreamWriter::append_all consumes a raw commitlog byte stream that must contain segment headers before any commit data. When the destination commitlog is empty (no current segment, per StreamWriter::create finding no existing segments) and commit bytes arrive before any segment marker, the writer returns io::ErrorKind::InvalidData 'no current segment, expected segment header'.

Solutions

1. Ensure the source stream starts at a segment boundary and includes the segment header (start streaming from the beginning of a segment)
2. If resuming into an existing destination repo, verify the repo actually contains the prior segment so a current segment exists
3. Regenerate the stream from a known-good source rather than patching the desynchronized one
4. Validate the first bytes of the stream against the '(ds)^2' magic before starting the writer

// Verify the stream begins with a segment header before handing it to StreamWriter
let mut peek = [0u8; 6];
let n = stream.peek(&mut peek).await?; // e.g. via a BufReader fill/peek helper
if n < 6 || &peek != b"(ds)^2" {
    return Err(io::Error::new(
        io::ErrorKind::InvalidData,
        "stream does not start with a segment header; restart the source at a segment boundary",
    ));
}

fn is_missing_segment_marker(err: &io::Error) -> bool {
    err.kind() == io::ErrorKind::InvalidData
        && err.to_string().contains("no current segment, expected segment header")
}

match writer.append_all(&mut stream, &mut progress).await {
    Ok(done) => Ok(done),
    Err(e) if is_missing_segment_marker(&e) => {
        // restart the source from a segment boundary, then retry with a fresh StreamWriter
        Err(e) // non-recoverable for this stream; caller must re-source it
    }
    Err(e) => Err(e),
}

When it happens

Trigger: Replaying a stream captured starting mid-segment (producer skipped the segment header); a stream whose leading segment marker was truncated or lost in transport; feeding arbitrary or desynchronized data into StreamWriter.

Common situations: Restarting network mirroring from a byte offset inside a segment instead of a segment boundary; a proxy/transport that dropped the first bytes; hand-built test streams that omit the header.

Related errors

• all-zeroes commit header
• repo : segment already exists and is non-empty
• expected commit offset
• failed to decode commit
• : log format version mismatch: current= segment=

AI-assisted analysis of clockworklabs/SpacetimeDB@9e0d92412f (2026-08-20). Data as JSON: /api/errors/c31db009ad8fdb6e. Report an issue: GitHub.

Appendix: source

Thrown at crates/commitlog/src/stream/writer.rs:233

                    let last_written_tx_range = self.last_written_tx_range.clone();
                    let commitlog_options = self.commitlog_options;
                    move || create_segment(repo, last_written_tx_range, commitlog_options, header)
                })
                .await
                .unwrap()
                .map(|(segment, index)| (segment.into_async_writer(), index))?;
                stream.consume(segment::Header::LEN as _);

                CurrentSegment {
                    header,
                    segment,
                    offset_index: index,
                }
            } else {
                match self.current_segment.take() {
                    Some(current_segment) => current_segment,
                    _ => {
                        return Err(io::Error::new(
                            io::ErrorKind::InvalidData,
                            "no current segment, expected segment header",
                        ));
                    }
                }
            };

            // What follows is commits to be written to `current_segment`,
            // until we encounter EOF or a segment marker.
            let res = self
                .append_all_inner(&mut stream, &mut current_segment, &mut progress)
                .await;
            // Ensure we flush application buffers (BufWriter).
            current_segment.segment.flush().await?;
            let maybe_eof = res?;
            // Put back segment, so it is available for syncing or closing.
            self.current_segment = Some(current_segment);
            match maybe_eof {

View on GitHub (pinned to 9e0d92412f)