{"record":{"id":"6525fb3e767f8ce1","repo":"clockworklabs/SpacetimeDB","slug":"failed-to-fsync-directory","errorCode":null,"errorMessage":"failed to fsync directory {}: {}","messagePattern":"failed to fsync directory (.+?): (.+?)","errorType":"exception","errorClass":"io::Error","httpStatus":null,"severity":"error","filePath":"crates/snapshot/src/lib.rs","lineNumber":1636,"sourceCode":"    /// On *nix systems, both a file and its enclosing directory should be\n    /// `fsync`ed to make the file durable.\n    ///\n    /// On Windows, only the file needs to be synced, and it's even an error to\n    /// sync a directory. Passing in [Self::Dir] is thus a no-op on Windows.\n    fn sync_all(&self) -> io::Result<()> {\n        match self {\n            #[cfg(target_os = \"windows\")]\n            Self::Dir(path) => Ok(()),\n            #[cfg(not(target_os = \"windows\"))]\n            Self::Dir(path) => File::open(path)\n                .map_err(|e| {\n                    io::Error::new(\n                        e.kind(),\n                        format!(\"failed to open directory {} for fsync: {}\", path.display(), e),\n                    )\n                })?\n                .sync_all()\n                .map_err(|e| io::Error::new(e.kind(), format!(\"failed to fsync directory {}: {}\", path.display(), e))),\n            Self::File(path) => {\n                File::options()\n                    .read(true)\n                    // Windows needs the file to be writable for `sync_all` to work.\n                    // Set all the open options explicitly, just for visibility.\n                    .write(true)\n                    .truncate(false)\n                    .create(false)\n                    .append(false)\n                    .open(path)\n                    .map_err(|e| {\n                        io::Error::new(\n                            e.kind(),\n                            format!(\"failed to open file {} for fsync: {}\", path.display(), e),\n                        )\n                    })?\n                    .sync_all()\n                    .map_err(|e| io::Error::new(e.kind(), format!(\"failed to fsync file {}: {}\", path.display(), e)))","sourceCodeStart":1618,"sourceCodeEnd":1654,"githubUrl":"https://github.com/clockworklabs/SpacetimeDB/blob/9e0d92412ff2248f401a8ad12d535f2b5ac30912/crates/snapshot/src/lib.rs#L1618-L1654","documentation":"After successfully opening a snapshot directory for fsync, crates/snapshot calls sync_all() on the directory File. If the OS-level fsync on the directory fails, the underlying io::Error kind is preserved and wrapped with 'failed to fsync directory {path}: {source}'. Directory fsync is what makes renames/creations inside it durable; a failure means the directory-entry changes may not survive a crash.","triggerScenarios":"Hardware/filesystem-level fsync errors (I/O error, failing disk); filesystems that do not support fsync on directory file descriptors (some FUSE/NFS configurations); interrupted underlying storage; ResourceExhausted conditions on some kernels under heavy sync load.","commonSituations":"Running on network or exotic filesystems (NFS, certain FUSE mounts, some container storage drivers) where directory fsync returns EINVAL/EIO; failing disks surfacing errors at barrier operations; test environments on overlayfs.","solutions":["Match the wrapped ErrorKind: for InvalidInput/Unsupported from exotic filesystems, treat directory fsync as best-effort and log-and-continue if your durability model permits","Move the snapshot store to a POSIX-compliant local filesystem (ext4/xfs) when directory durability is required","Investigate hardware/kernel logs (dmesg) if EIO — the storage itself is unhealthy","Do not silently swallow: at minimum record which directory failed so recovery can re-verify"],"exampleFix":"// before\nSyncTarget::Dir(&dir).sync_all()?; // EIO/EINVAL from FUSE mount\n\n// after (best-effort directory fsync on exotic filesystems)\nif let Err(e) = SyncTarget::Dir(&dir).sync_all() {\n    tracing::warn!(?dir, %e, \"directory fsync failed; durability of dir entries not guaranteed\");\n}","handlingStrategy":"try-catch","validationCode":"// Nothing to validate at the API level; check the mount supports it once at startup:\nlet probe = File::open(&snapshot_dir)?.sync_all();\nif probe.is_err() { tracing::warn!(\"directory fsync unsupported on this mount\"); }","typeGuard":null,"tryCatchPattern":"if let Err(e) = SyncTarget::Dir(&dir).sync_all() {\n    match e.kind() {\n        std::io::ErrorKind::Unsupported | std::io::ErrorKind::InvalidInput => {\n            tracing::warn!(?dir, %e, \"dir fsync unsupported; durability best-effort\");\n        }\n        _ => return Err(e),\n    }\n}","preventionTips":["Prefer local filesystems (ext4/xfs) for durability-critical snapshot stores","Investigate EIO-class failures in kernel logs before continuing","Log every directory-fsync failure with the path for post-crash auditing"],"tags":["filesystem","fsync","durability","rust","snapshot","spacetimedb"],"backgroundTag":"fsync-failed","analyzedSha":"9e0d92412ff2248f401a8ad12d535f2b5ac30912","analyzedAt":"2026-08-20T06:08:37.179Z","contentChangedAt":"2026-08-20T06:08:37.179Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}