{"record":{"id":"c4ca8bc77b0f35af","repo":"clockworklabs/SpacetimeDB","slug":"fsync-failed","errorCode":null,"errorMessage":"fsync failed","messagePattern":"fsync failed","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"crates/commitlog/src/stream/common.rs","lineNumber":48,"sourceCode":"    fn open_segment_reader_async(\n        &self,\n        offset: u64,\n    ) -> impl Future<Output = io::Result<Self::AsyncSegmentReader>> + Send;\n}\n\npub trait AsyncFsync {\n    fn fsync(&self) -> impl Future<Output = ()> + Send;\n}\n\nimpl<T: AsyncWrite + AsyncFsync + Send + Sync> AsyncFsync for tokio::io::BufWriter<T> {\n    async fn fsync(&self) {\n        self.get_ref().fsync().await\n    }\n}\n\nimpl AsyncFsync for tokio::fs::File {\n    async fn fsync(&self) {\n        self.sync_data().await.expect(\"fsync failed\")\n    }\n}\n\npub trait AsyncLen: AsyncSeek + Unpin + Send {\n    fn segment_len(&mut self) -> impl Future<Output = io::Result<u64>> + Send\n    where\n        Self: Sized,\n    {\n        async { spacetimedb_fs_utils::compression::segment_len(self).await }\n    }\n}\n\nimpl<T: AsyncWrite + AsyncLen + Send> AsyncLen for tokio::io::BufWriter<T> {\n    async fn segment_len(&mut self) -> io::Result<u64> {\n        self.get_mut().segment_len().await\n    }\n}\n","sourceCodeStart":30,"sourceCodeEnd":66,"githubUrl":"https://github.com/clockworklabs/SpacetimeDB/blob/524b4487d949b61a07d4f39c862d1290259dfd20/crates/commitlog/src/stream/common.rs#L30-L66","documentation":"`AsyncFsync` is SpacetimeDB's internal trait for durability on async files; the impl for `tokio::fs::File` calls `sync_data().await.expect(\"fsync failed\")`, so any error from the OS fsync aborts the task. An fsync error means the OS could not guarantee data reached stable storage — typically ENOSPC, EIO, or a vanished/failed backend device.","triggerScenarios":"Any code path that fsyncs a commitlog segment or other async-written file on a full, failing, or detached filesystem: quota exhausted mid-write, dm-crypt/NFS backend returning EIO, or the block device being removed underneath the process.","commonSituations":"Containers with small ephemeral volumes; data directories placed on NFS/SMB or FUSE filesystems with unreliable fsync semantics; failing SSDs or HW RAID members; ENOSPC after large burst writes.","solutions":["Free or expand space on the affected filesystem (df -h / container limits) — ENOSPC is the most frequent trigger.","Check kernel logs (dmesg, journalctl -k) for EIO or device errors; test with smartctl and replace faulty hardware.","Move the database data directory off NFS/FUSE onto local ext4/xfs storage.","Restart the process after fixing the underlying cause; already-open files may be in an inconsistent state."],"exampleFix":"// before: the trait impl panics on any fsync error\nasync fn fsync(&self) { self.sync_data().await.expect(\"fsync failed\") }\n\n// after (if you control the call site): propagate instead of panicking\nasync fn fsync_checked(file: &tokio::fs::File) -> io::Result<()> {\n    file.sync_data().await\n}","handlingStrategy":"validation","validationCode":"// Preflight free space before heavy writes (nix crate statvfs on the data dir);\n// abort writes when available bytes fall below one segment + headroom.","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Provision data volumes with headroom and monitor usage.","Use local filesystems with reliable fsync (ext4, xfs); avoid NFS/FUSE for databases.","Surface SMART/RAID health alerts so hardware EIO is caught before fsync fails."],"tags":["rust","tokio","fsync","disk-io","spacetimedb"],"backgroundTag":"disk-fsync-failure","analyzedSha":"524b4487d949b61a07d4f39c862d1290259dfd20","analyzedAt":"2026-08-16T23:58:54.611Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}