{"record":{"id":"f7a4dfcc854ef681","repo":"tracel-ai/burn","slug":"failed-to-spawn-session-worker-thread","errorCode":null,"errorMessage":"Failed to spawn session worker thread","messagePattern":"Failed to spawn session worker thread","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/burn-remote/src/server/worker.rs","lineNumber":126,"sourceCode":"            transfer,\n            local_comm,\n            graphs: Mutex::new(HashMap::new()),\n            probe,\n        };\n        let (sender, receiver) = mpsc::channel(TASK_CHANNEL_CAPACITY);\n        handler.drive(receiver);\n        sender\n    }\n\n    // Must be called from within the runtime that owns the endpoint: it captures `Handle::current`.\n    #[cfg(not(target_family = \"wasm\"))]\n    fn drive(self, receiver: mpsc::Receiver<Task>) {\n        let handle = Handle::current();\n        let session_id = self.session_id;\n        std::thread::Builder::new()\n            .name(format!(\"burn-remote-session-{session_id}\"))\n            .spawn(move || handle.block_on(self.run(receiver)))\n            .expect(\"Failed to spawn session worker thread\");\n    }\n\n    #[cfg(target_family = \"wasm\")]\n    fn drive(self, receiver: mpsc::Receiver<Task>) {\n        spawn_detached(self.run(receiver));\n    }\n\n    /// Drain the task channel, running each task to completion in arrival order, then tear the\n    /// session down in an order that releases its memory.\n    async fn run(mut self, mut receiver: mpsc::Receiver<Task>) {\n        let session_id = self.session_id;\n\n        log::debug!(\"Session {session_id} worker started\");\n        while let Some(task) = receiver.recv().await {\n            if let Err(err) = self.process_task(task).await {\n                // One task failing doesn't tear down the session: read/sync/dtype failures surface\n                // to the client through their response, fire-and-forget failures are logged here,\n                // and the worker keeps processing subsequent tasks.","sourceCodeStart":108,"sourceCodeEnd":144,"githubUrl":"https://github.com/tracel-ai/burn/blob/d16f7ba2ed0d41408189384044cc886fb4c8f957/crates/burn-remote/src/server/worker.rs#L108-L144","documentation":"The session worker spawns a dedicated OS thread per remote session that block_on's the session's async run loop. The expect panics if std::thread::Builder::spawn fails, i.e. the OS refused to create a new thread.","triggerScenarios":"Creating a new remote session when the process/system cannot spawn a thread — thread-count limits (ulimit, cgroup pids.max), memory exhaustion, or too many concurrent sessions.","commonSituations":"Many simultaneous remote clients each opening a session on a constrained container; hitting RLIMIT_NPROC; leaks of previous session threads exhausting resources.","solutions":["Reduce concurrent sessions or raise the thread/pids limit (ulimit -u, cgroup pids.max)","Fix leaks: ensure sessions are closed so their worker threads exit","Increase container memory; thread stack allocation may fail under low memory","Patch/handle spawn failure gracefully instead of expect if graceful degradation is desired"],"exampleFix":"// before\n.spawn(move || handle.block_on(self.run(receiver)))\n.expect(\"Failed to spawn session worker thread\");\n// after\n// system-level: raise limits, e.g. `ulimit -u 4096` or set pids.max in the cgroup","handlingStrategy":"retry","validationCode":"// Sanity-check spawn capacity before opening sessions\nstd::thread::Builder::new().name(\"probe\").spawn(|| {}).map_err(|e| anyhow!(\"cannot spawn threads: {e}\"))?;","typeGuard":null,"tryCatchPattern":"match std::thread::Builder::new().name(name).spawn(work) {\n    Ok(h) => h,\n    Err(e) => return Err(anyhow!(\"session worker spawn failed: {e}\")),\n}","preventionTips":["Cap concurrent remote sessions; each spawns a dedicated thread","Raise RLIMIT_NPROC / cgroup pids.max for the server process","Ensure sessions are properly shut down so worker threads don't leak"],"tags":["rust","threads","resources","server"],"backgroundTag":"thread-spawn-failed","analyzedSha":"d16f7ba2ed0d41408189384044cc886fb4c8f957","analyzedAt":"2026-09-05T13:19:14.260Z","contentChangedAt":"2026-09-05T13:19:14.260Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}