clockworklabs/SpacetimeDB · error · std::io::Error

Failed to truncate offset index: {e}

Error message

Failed to truncate offset index: {e}

What it means

Raised during Commitlog::truncate: after a segment file is cut back to a given commit offset, the matching offset-index file must also be truncated (to offset + 1, since the retained commit stays indexed). If index_file.ftruncate fails, the error is wrapped as InvalidData with this message, leaving the index and segment potentially out of sync until truncation is retried.

Source

Thrown at crates/commitlog/src/commitlog.rs:742

                    break;
                }
                byte_offset += Commit::from(commit).encoded_len() as u64;
            }

            if byte_offset == segment::Header::LEN as u64 {
                // Segment is empty, just remove it.
                repo.remove_segment(segment)?;
            } else {
                debug!("truncating segment {segment} to {offset} at {byte_offset}");
                let mut file = repo.open_segment_writer(segment)?;

                if let Some(mut index_file) = index_file {
                    let index_file = index_file.as_mut();
                    // Note: The offset index truncates equal or greater,
                    // inclusive. We'd like to retain `offset` in the index, as
                    // the commit is also retained in the log.
                    index_file.ftruncate(offset + 1, byte_offset).map_err(|e| {
                        io::Error::new(
                            io::ErrorKind::InvalidData,
                            format!("Failed to truncate offset index: {e}"),
                        )
                    })?;
                    index_file.async_flush()?;
                }

                file.ftruncate(offset, byte_offset)?;
                // We should be reopening the log anyway, but just in case.
                file.seek(io::SeekFrom::End(0))?;
                // Some filesystems require fsync after ftruncate.
                file.fsync()?;
                break;
            }
        }
    }

    Ok(())

View on GitHub (pinned to 9e0d92412f)

Solutions

1. Fix the filesystem condition: free space, remount read-write, restore ownership/permissions on the index files.
2. Retry the truncation/recovery once the data directory is healthy.
3. Ensure nothing else modifies or removes index files while the node runs.

if let Err(e) = commitlog.truncate(offset, byte_offset) {
    if e.kind() == io::ErrorKind::InvalidData && e.to_string().contains("offset index") {
        // repair filesystem (space/permissions) then retry the truncation once
    } else {
        return Err(e.into());
    }
}

When it happens

Trigger: Truncating a commitlog when the offset-index truncate fails: permissions changed on the index file, the file was deleted by another process, the volume went read-only or full, or an IO error occurred during ftruncate.

Common situations: Recovery after a crash while something else (backup job, antivirus, ops script) touches the data directory; full or read-only disks during recovery.

Related errors

• failed to read {} bytes of commit payload: {}
• error reading commit header: {e}
• failed to flush segment upon rotation
• repo {}: error getting file metadata for segment {}: {}
• mismatched key in offset index file

AI-assisted analysis of clockworklabs/SpacetimeDB@9e0d92412f (2026-08-20). Data as JSON: /api/errors/3782b4997469e342. Report an issue: GitHub.