{"record":{"id":"4570441bf62c70fa","repo":"spacedriveapp/spacedrive","slug":"worker-channel-closed-trying-to-force-task-abortio","errorCode":null,"errorMessage":"Worker channel closed trying to force task abortion","messagePattern":"Worker channel closed trying to force task abortion","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/task-system/src/worker/mod.rs","lineNumber":178,"sourceCode":"\t\t&self,\n\t\ttask_id: TaskId,\n\t\tack: oneshot::Sender<Result<(), SystemError>>,\n\t) {\n\t\tself.msgs_tx\n\t\t\t.send(WorkerMessage::CancelNotRunningTask { task_id, ack })\n\t\t\t.await\n\t\t\t.expect(\"Worker channel closed trying to cancel a not running task\");\n\t}\n\n\tpub async fn force_task_abortion(\n\t\t&self,\n\t\ttask_id: TaskId,\n\t\tack: oneshot::Sender<Result<(), SystemError>>,\n\t) {\n\t\tself.msgs_tx\n\t\t\t.send(WorkerMessage::ForceAbortion { task_id, ack })\n\t\t\t.await\n\t\t\t.expect(\"Worker channel closed trying to force task abortion\");\n\t}\n\n\t#[instrument(skip(self), fields(worker_id = self.id))]\n\tpub async fn shutdown(&self) {\n\t\tif let Some(handle) = self\n\t\t\t.handle\n\t\t\t.try_borrow_mut()\n\t\t\t.ok()\n\t\t\t.and_then(|mut maybe_handle| maybe_handle.take())\n\t\t{\n\t\t\tlet (tx, rx) = oneshot::channel();\n\n\t\t\tself.msgs_tx\n\t\t\t\t.send(WorkerMessage::ShutdownRequest(tx))\n\t\t\t\t.await\n\t\t\t\t.expect(\"Worker channel closed trying to shutdown\");\n\n\t\t\trx.await.expect(\"Worker channel closed trying to shutdown\");","sourceCodeStart":160,"sourceCodeEnd":196,"githubUrl":"https://github.com/spacedriveapp/spacedrive/blob/6dfeccf2113039e35f2ce735f945e70dc3e4ea45/crates/task-system/src/worker/mod.rs#L160-L196","documentation":"Worker::force_task_abortion sends a ForceAbortion message over the worker's command channel and panics with .expect() when the channel is closed, i.e. the worker's message-processing task has exited and dropped its receiver. It means a force-abort was directed at a worker that has already shut down, so the request can never be acknowledged.","triggerScenarios":"The dispatch chain system.rs:396/408 -> Worker::force_task_abortion (worker/mod.rs:170) firing after the target worker's run loop returned from ShutdownRequest handling (run.rs:90-92), or while the runtime is being torn down and the worker task is dropped. Note force_task_abortion is await-heavy on the worker side (run.rs:79-88 awaits runner.force_task_abortion), so a worker blocked there while the system shuts down increases the race window.","commonSituations":"Force-aborting stuck tasks exactly during shutdown (the most common time to force-abort); watchdog code that aborts tasks on a timer firing concurrently with teardown; tests that force-abort then immediately shutdown the system without awaiting in between.","solutions":["Sequence force-aborts before shutdown begins, and await them (the ack oneshot) before calling system.shutdown()","Library-level fix: map SendError to a SystemError (e.g. worker-unavailable) or warn-and-return instead of .expect, since shutdown already force-finalizes all tasks the worker owned","Ensure your tasks honor interrupt signals (Interrupter/worktable) so force-abort acks return quickly and don't overlap the shutdown window","As a last-resort containment, call force-abort through AssertUnwindSafe(..).catch_unwind() and interpret a panic as 'worker gone, nothing to abort'"],"exampleFix":"// before (worker/mod.rs:175-178)\nself.msgs_tx\n    .send(WorkerMessage::ForceAbortion { task_id, ack })\n    .await\n    .expect(\"Worker channel closed trying to force task abortion\");\n\n// after\nif self\n    .msgs_tx\n    .send(WorkerMessage::ForceAbortion { task_id, ack })\n    .await\n    .is_err()\n{\n    warn!(%task_id, worker_id = self.id, \"Worker channel closed; force abortion request dropped (worker already shutdown)\");\n}","handlingStrategy":"validation","validationCode":"use std::sync::atomic::{AtomicBool, Ordering};\n\nstatic SHUTTING_DOWN: AtomicBool = AtomicBool::new(false);\n\nfn can_force_abort() -> bool { !SHUTTING_DOWN.load(Ordering::Acquire) }\n\n// Force-abort is await-heavy on the worker side (run.rs:79-88); complete it\n// BEFORE shutdown begins, otherwise skip it and let shutdown finalize the task.","typeGuard":null,"tryCatchPattern":"use futures::FutureExt;\nuse std::panic::AssertUnwindSafe;\n\nif AssertUnwindSafe(remote.force_abort()).catch_unwind().await.is_err() {\n    // Worker already down; its shutdown path force-finalizes every task it owned\n    // (send_forced_abortion_task_response, runner.rs:1385-1400).\n}","preventionTips":["Design tasks to honor the Interrupter/worktable interrupt signals so force-abort is rarely needed during teardown","Never overlap force-abort with system.shutdown(); sequence them and await the ack in between","Treat force-abort-after-shutdown as a no-op by contract, since shutdown already finalizes all tasks"],"tags":["rust","tokio","async","panic","channel","shutdown","race-condition","task-system"],"backgroundTag":null,"analyzedSha":"6dfeccf2113039e35f2ce735f945e70dc3e4ea45","analyzedAt":"2026-08-16T11:26:17.074Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}