{"record":{"id":"ad575ee05faa8ede","repo":"spacedriveapp/spacedrive","slug":"stolen-task-channel-closed","errorCode":null,"errorMessage":"Stolen task channel closed","messagePattern":"Stolen task channel closed","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/task-system/src/worker/mod.rs","lineNumber":292,"sourceCode":"\t\t\t.skip(stealer_id)\n\t\t\t// Taking the total amount of workers\n\t\t\t.take(total_workers)\n\t\t\t// Removing the current worker as we can't steal from ourselves\n\t\t\t.filter(|worker_comm| worker_comm.worker_id != stealer_id)\n\t\t{\n\t\t\tif worker_comm\n\t\t\t\t.steal_task(stealer_id, stolen_task_tx.clone())\n\t\t\t\t.await\n\t\t\t{\n\t\t\t\ttrace!(stolen_worker_id = worker_comm.worker_id, \"Stole a task\");\n\t\t\t\treturn;\n\t\t\t}\n\t\t}\n\n\t\tstolen_task_tx\n\t\t\t.send(None)\n\t\t\t.await\n\t\t\t.expect(\"Stolen task channel closed\");\n\t}\n}\n","sourceCodeStart":274,"sourceCodeEnd":295,"githubUrl":"https://github.com/spacedriveapp/spacedrive/blob/6dfeccf2113039e35f2ce735f945e70dc3e4ea45/crates/task-system/src/worker/mod.rs#L274-L295","documentation":"WorkStealer::steal ends by sending None ('nothing was stolen') back to the requesting worker's stole_task_tx and panics via .expect() if that channel is closed. The receiver, stole_task_rx, lives inside the requesting worker's merged message stream (run.rs:40-46); when that run future dies without aborting the detached steal task, the stream is dropped, the channel closes, and the terminal send at mod.rs:289-292 panics inside the spawned steal task.","triggerScenarios":"The requesting worker's run future panicking and the respawn loop restarting it (mod.rs:69-83): the old msg_stream holding stole_task_rx is dropped, but the steal task spawned at runner.rs:1402-1407 is only aborted via abort_steal_task() on the guarded paths (NewTask at run.rs:53 and shutdown at runner.rs:577) - the panic-restart path drops the Runner (which owns current_steal_task_handle) without aborting, so the detached steal task survives just long enough to panic on send(None). Also possible during runtime teardown.","commonSituations":"Any worker panic-restart while an idle steal sweep is in flight (steals fire every second per idle worker, runner.rs:917-929, so the overlap window is large in mostly-idle pools); production logs showing an stray panic from a detached task right after a 'Worker critically failed and will restart' line.","solutions":["Library-level fix: make the terminal send non-fatal - let _ = stolen_task_tx.send(None).await; - a closed receiver means the requester is gone and the None signal is moot","Library-level fix: abort the outstanding steal task on every exit path from run, not just NewTask/shutdown - e.g. a guard in run() (or Drop on Runner) that calls abort_steal_task(), or move current_steal_task_handle somewhere that survives into the respawn loop","Fix the underlying worker panic that triggers the restart (see the 'Worker critically failed' log) - this error is a secondary symptom","Keep runtime/task lifetimes stable so run futures are never dropped mid-sweep"],"exampleFix":"// before (worker/mod.rs:289-292)\nstolen_task_tx\n    .send(None)\n    .await\n    .expect(\"Stolen task channel closed\");\n\n// after\nif stolen_task_tx.send(None).await.is_err() {\n    trace!(stealer_id, \"Requesting worker gone before 'no task stolen' signal; nothing to do\");\n}","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Panic fires in the detached WorkStealer::steal task; catch it with a hook:\nstd::panic::set_hook(Box::new(|info| {\n    let msg = info.to_string();\n    if msg.contains(\"Stolen task channel closed\") {\n        // requester's run loop died mid-sweep; the None signal is moot\n        return;\n    }\n    eprintln!(\"panic: {msg}\");\n}));","preventionTips":["Eliminate worker panics: this only fires when a worker's run loop is dropped/restarted with an idle steal sweep still in flight (the guarded paths, NewTask and shutdown, already abort the steal task)","Run with panic=unwind and fix every 'Worker critically failed and will restart' occurrence - each restart is a chance for this secondary panic","Avoid dropping/cancelling the task system's tasks from outside; drive teardown exclusively through system.shutdown()"],"tags":["rust","tokio","async","panic","channel","work-stealing","detached-task","task-system"],"backgroundTag":null,"analyzedSha":"6dfeccf2113039e35f2ce735f945e70dc3e4ea45","analyzedAt":"2026-08-16T11:26:17.074Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}