clockworklabs/SpacetimeDB · error · io::Error

no current segment, expected segment header

Error message

no current segment, expected segment header

What it means

StreamWriter::append_all ingests a replication/bootstrap stream into the local commitlog. A segment is only set up when the stream yields a buffer starting with the segment magic (a segment header); otherwise data is appended to the latest segment found when the writer was created. This error means neither was available: the stream's next chunk was not a segment header AND the writer has no current segment to append to (the target repo was empty at create time).

Source

Thrown at crates/commitlog/src/stream/writer.rs:233

                    let last_written_tx_range = self.last_written_tx_range.clone();
                    let commitlog_options = self.commitlog_options;
                    move || create_segment(repo, last_written_tx_range, commitlog_options, header)
                })
                .await
                .unwrap()
                .map(|(segment, index)| (segment.into_async_writer(), index))?;
                stream.consume(segment::Header::LEN as _);

                CurrentSegment {
                    header,
                    segment,
                    offset_index: index,
                }
            } else {
                match self.current_segment.take() {
                    Some(current_segment) => current_segment,
                    _ => {
                        return Err(io::Error::new(
                            io::ErrorKind::InvalidData,
                            "no current segment, expected segment header",
                        ));
                    }
                }
            };

            // What follows is commits to be written to `current_segment`,
            // until we encounter EOF or a segment marker.
            let res = self
                .append_all_inner(&mut stream, &mut current_segment, &mut progress)
                .await;
            // Ensure we flush application buffers (BufWriter).
            current_segment.segment.flush().await?;
            let maybe_eof = res?;
            // Put back segment, so it is available for syncing or closing.
            self.current_segment = Some(current_segment);
            match maybe_eof {

View on GitHub (pinned to 524b4487d9)

Solutions

1. Start the stream at offset 0 (or at a segment boundary) so a segment header precedes the commits.
2. Re-create the writer on a repo that still contains the segment being continued, so the current segment is populated.
3. Align producer and consumer on the same commitlog crate/format version.

Example fix

// before: feeding a mid-segment stream into a fresh repo
let writer = StreamWriter::create(repo, opts, OnTrailingData::Error)?;
let writer = writer.append_all(stream_from_offset(mid_segment_tx), progress).await?; // Err: no current segment

// after: begin at the segment start so the header arrives first
let writer = StreamWriter::create(repo, opts, OnTrailingData::Error)?;
let writer = writer.append_all(stream_from_offset(0), progress).await?;

use tokio::io::{AsyncBufRead, AsyncBufReadExt};

async fn stream_starts_with_segment_header<S: AsyncBufRead + Unpin>(
    stream: &mut S,
) -> std::io::Result<bool> {
    let buf = stream.fill_buf().await?;
    Ok(buf.starts_with(&segment::MAGIC))
}

match writer.append_all(stream, progress).await {
    Err(e) if e.kind() == io::ErrorKind::InvalidData && e.to_string().contains("expected segment header") => {
        // Restart the feed from offset 0 / a segment boundary and re-create the writer.
    }
    r => r,
}

When it happens

Trigger: Calling StreamWriter::append_all with a stream that begins mid-segment (no leading segment header) against an empty or newly created commitlog repo; resuming a stream from a mid-segment offset after the local log was wiped; a producer that skips the header, or a framing/protocol version mismatch between producer and consumer.

Common situations: Follower re-bootstrap into a cleared data directory while the leader starts streaming from a non-zero offset; test harnesses feeding hand-crafted commit streams; producer and consumer built from different commitlog format versions.

Related errors

• failed to decode commit
• all-zeroes commit header
• expected commit offset {} but encountered {}
• failed to flush segment upon rotation
• never types are not yet supported in C# output

AI-assisted analysis of clockworklabs/SpacetimeDB@524b4487d9 (2026-08-16). Data as JSON: /api/errors/c31db009ad8fdb6e. Report an issue: GitHub.