{"record":{"id":"44a4001a74a8f632","repo":"clockworklabs/SpacetimeDB","slug":"error-syncing-data-to-disk","errorCode":null,"errorMessage":"error syncing data to disk","messagePattern":"error syncing data to disk","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/core/src/database_logger.rs","lineNumber":518,"sourceCode":"        // If following isn't requested, we can stream the data.\n        if !follow {\n            return self.logger.lock().await.tail_stream(n);\n        }\n        match n {\n            // If we don't need to access the disk,\n            // locking and spawning can be avoided.\n            None | Some(0) => self.subscribe().map(Ok).boxed(),\n\n            // Otherwise, we need to hold the lock to prevent writes\n            // while we gather a snapshot of the persistent tail.\n            Some(n) => {\n                // Cap reading the tail into memory at a few hundred KiB.\n                let n = n.min(2500);\n                let (tail, more) = {\n                    let inner = self.logger.clone().lock_owned().await;\n                    let more = self.subscribe();\n                    asyncify(move || {\n                        inner.sync_data().expect(\"error syncing data to disk\");\n                        (inner.tail(n), more)\n                    })\n                }\n                .await;\n\n                stream::once(future::ready(tail)).chain(more.map(Ok)).boxed()\n            }\n        }\n    }\n\n    fn subscribe(&self) -> impl Stream<Item = Bytes> + use<T> {\n        BroadcastStream::new(self.broadcast.subscribe()).filter_map(move |x| {\n            future::ready(match x {\n                Ok(chunk) => Some(chunk),\n                Err(BroadcastStreamRecvError::Lagged(skipped)) => {\n                    log::trace!(\"skipped {skipped} lines in module log\");\n                    None\n                }","sourceCodeStart":500,"sourceCodeEnd":536,"githubUrl":"https://github.com/clockworklabs/SpacetimeDB/blob/6dee26c6efc2856793e12b148a59742964f5d783/crates/core/src/database_logger.rs#L500-L536","documentation":"Panic in `DatabaseLogger::tail`/stream logic: when a caller requests a bounded tail (`n = Some(>0)`) of the database log, the implementation locks the log file and calls `inner.sync_data().expect(\"error syncing data to disk\")` inside a blocking thread before snapshotting the last lines. If `sync_data` fails (I/O error, file closed), the panic fires inside `asyncify`, killing the task that requested the log stream.","triggerScenarios":"Calling `tail(Some(n), ..)` / subscribing to a bounded database-log tail while the underlying log file is on failing storage or was closed/truncated externally. With `None` or `Some(0)` the sync path is skipped entirely, so only bounded tails hit it.","commonSituations":"Reading recent database logs (e.g. `spacetime logs --last N` or energy/metrics dashboards tailing logs) right as a disk error or container volume issue occurs; file descriptor invalidated by a container restart; quota exhaustion during sync. Usually co-occurs with errors 604/605 from the same storage fault.","solutions":["Treat it as a storage-fault symptom: check `dmesg`/disk SMART, free space, and mount health on the database directory, then restart the node.","Retry the log tail after remediating storage — the panic is per-request, so unlike the commit log poison flag it does not permanently corrupt state.","If you only need live logs, request the unbounded/follow-only variant (`n = None`) which skips the `sync_data` path entirely.","For embedded use, catch the panic boundary (`futures::FutureExt::catch_unwind`) around log-tail streams so a sync failure degrades to stale logs instead of a crashed task."],"exampleFix":"// before: bounded tail triggers sync_data and can panic\nlogger.tail(Some(100), true).await\n\n// after: follow-only stream avoids the sync_data path\nlogger.tail(None, true).await","handlingStrategy":"validation","validationCode":"// Avoid the sync path entirely when live-following: pass unbounded n\nlet stream = db_logger.tail(None, true).await?; // skips sync_data entirely","typeGuard":null,"tryCatchPattern":"let fut = logger.tail(Some(n), false);\nlet r = future::FutureExt::catch_unwind(std::pin::pin!(fut));\nif r.await.is_err() { /* fall back to None (follow-only) which never syncs */ }","preventionTips":["Prefer follow-only tails (n = None) unless you truly need a bounded history snapshot.","Monitor storage health on the database log directory.","Wrap log-tail streams in catch_unwind in dashboards so a sync fault degrades instead of crashing the UI."],"tags":["rust","spacetimedb","logging","fsync","disk-io","tail","panic"],"backgroundTag":"fsync-failed","analyzedSha":"6dee26c6efc2856793e12b148a59742964f5d783","analyzedAt":"2026-08-20T06:08:37.179Z","contentChangedAt":"2026-08-20T06:08:37.179Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}