{"record":{"id":"b60939dcbaa0d329","repo":"clockworklabs/SpacetimeDB","slug":"log-worker-panicked","errorCode":null,"errorMessage":"log worker panicked","messagePattern":"log worker panicked","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/core/src/database_logger.rs","lineNumber":380,"sourceCode":"            LogLevel::Info => LogEvent::Info(record),\n            LogLevel::Debug => LogEvent::Debug(record),\n            LogLevel::Trace => LogEvent::Trace(record),\n            LogLevel::Panic => {\n                trace = bt.capture();\n                frames = trace.frames();\n                LogEvent::Panic { record, trace: &frames }\n            }\n        };\n        // TODO(perf): Reuse serialization buffer.\n        let mut buf = serde_json::to_string(&event).unwrap();\n        buf.push('\\n');\n        let buf = Bytes::from(buf);\n        self.cmd\n            .send(Cmd::Append {\n                ts: record.ts,\n                record: buf,\n            })\n            .expect(\"log worker panicked\");\n    }\n\n    /// Stream the contents of this logger.\n    ///\n    /// If `n` is `Some`, only yield up to the last `n` lines in the log.\n    /// If `follow` is `true`, the stream waits for new records to be appended\n    /// to the log (via [Self::write]) and yields them as they become available.\n    pub async fn tail(&self, n: Option<u32>, follow: bool) -> Result<LogStream, LoggerPanicked> {\n        let (tx, rx) = oneshot::channel();\n        self.cmd.send(Cmd::Tail { n, follow, reply: tx })?;\n        Ok(rx.await?)\n    }\n\n    /// Read the most recent logs in `logs_dir`, up to `num_lines`.\n    ///\n    /// Note that this only reads from the most recent log file, even if it\n    /// contains less than `num_lines` lines.\n    ///","sourceCodeStart":362,"sourceCodeEnd":398,"githubUrl":"https://github.com/clockworklabs/SpacetimeDB/blob/524b4487d949b61a07d4f39c862d1290259dfd20/crates/core/src/database_logger.rs#L362-L398","documentation":"`DatabaseLogger::write` forwards each serialized log record over an mpsc channel to a dedicated worker task; `.expect(\"log worker panicked\")` fires when `send` fails, which only happens once the receiving worker has exited. In practice the worker itself panicked earlier (look for the first panic in stderr) or the logger is being torn down while a write is still in flight.","triggerScenarios":"Calling `logger.write(...)` (or any module log path) after the worker task terminated — the worker hit an fsync or serialization panic, or the DatabaseLogger/runtime is dropped concurrently with a write during shutdown.","commonSituations":"Shutdown races while stopping spacetimedb where module log writes race logger teardown; an earlier disk error killed the worker; tests dropping the tokio runtime while modules still emit logs.","solutions":["Find the FIRST panic in process output — the worker's original error explains why the channel closed; this expect is only the downstream symptom.","Fix that root cause (commonly the fsync/disk error or serialization failure that killed the worker).","Ensure module hosts stop producing log records before the logger/runtime is dropped during shutdown.","Upgrade spacetimedb if a known worker-panic bug exists for your version."],"exampleFix":"// before\nself.cmd.send(Cmd::Append { ts: record.ts, record: buf }).expect(\"log worker panicked\");\n\n// after: tolerate a dead worker during teardown instead of aborting\nif self.cmd.send(Cmd::Append { ts: record.ts, record: buf }).is_err() {\n    eprintln!(\"[database_logger] worker unavailable: {}\", String::from_utf8_lossy(&buf));\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"let r = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| logger.write(record))); if r.is_err() { /* worker died: find the original panic in stderr, stop logging to this sink */ }","preventionTips":["Keep the DatabaseLogger alive for the whole process lifetime; shut down log producers before dropping it.","Investigate the first panic in the output, not this one — it is downstream of the worker's death.","Avoid writing logs from tasks that may outlive the runtime."],"tags":["rust","spacetimedb","logging","channel-closed","task-panic"],"backgroundTag":"background-task-panicked","analyzedSha":"524b4487d949b61a07d4f39c862d1290259dfd20","analyzedAt":"2026-08-16T23:58:54.611Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}