clockworklabs/SpacetimeDB · error · std::io::Error

segment {offset} already exists

Error message

segment {offset} already exists

What it means

In-memory counterpart of the filesystem 'segment already exists' error: Memory::create_segment looks the offset up in its BTreeMap and only recycles an entry that is empty. If a non-empty in-memory segment already exists at that offset, creation fails with AlreadyExists. Because Memory repos back tests and ephemeral stores, this almost always indicates offset bookkeeping bugs in test or setup code.

Source

Thrown at crates/commitlog/src/repo/mem.rs:66

impl fmt::Display for Memory {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        f.write_str("<memory>")
    }
}

impl Repo for Memory {
    type SegmentWriter = Segment;
    type SegmentReader = ReadOnlySegment;

    fn create_segment(&self, offset: u64, header: crate::segment::Header) -> io::Result<Self::SegmentWriter> {
        let mut inner = self.segments.write().unwrap();
        let mut segment = match inner.entry(offset) {
            btree_map::Entry::Occupied(entry) => {
                let entry = entry.get();
                if entry.read().unwrap().is_empty() {
                    Segment::from_shared(self.space.clone(), entry.clone())
                } else {
                    return Err(io::Error::new(
                        io::ErrorKind::AlreadyExists,
                        format!("segment {offset} already exists"),
                    ));
                }
            }
            btree_map::Entry::Vacant(entry) => {
                let storage = entry.insert(Arc::new(RwLock::new(Storage::new())));
                Segment::from_shared(self.space.clone(), storage.clone())
            }
        };
        header.write(&mut segment)?;

        Ok(segment)
    }

    fn open_segment_writer(&self, offset: u64) -> io::Result<Self::SegmentWriter> {
        let inner = self.segments.read().unwrap();
        let Some(buf) = inner.get(&offset) else {

View on GitHub (pinned to 6dee26c6ef)

Solutions

1. Create a fresh Memory repo (or clear its segments) between test cases that reuse offsets.
2. Derive new segment offsets from the current head's min_tx_offset plus its commit count, not from a constant.
3. If the segment legitimately exists, open it with open_segment_writer instead of creating.

Example fix

// before
let repo = Memory::default();
repo.create_segment(0, hdr)?;
repo.create_segment(0, hdr)?; // AlreadyExists

// after
let repo = Memory::default();
let seg = repo.create_segment(0, hdr)?;
// append commits, then open (not create) when revisiting offset 0
let seg = repo.open_segment_writer(0)?;

match repo.create_segment(off, hdr) {
    Ok(seg) => seg,
    Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => repo.open_segment_writer(off)?,
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Calling create_segment(offset, ...) twice at the same offset on a Memory repo where the first segment received data; deriving offsets from stale head state in a test harness.

Common situations: Unit tests that re-initialize a shared Memory repo without resetting it; helper code with off-by-one segment-offset arithmetic that collides with the previous segment.

Related errors

• segment {offset} does not exist
• refusing to compress mutable segment {head_offset}
• invalid seek to a negative or overflowing position
• no space left on device
• repo {}: too many empty segments: {}

AI-assisted analysis of clockworklabs/SpacetimeDB@6dee26c6ef (2026-08-20). Data as JSON: /api/errors/4aca06ee8fc50b8a. Report an issue: GitHub.