{"record":{"id":"9767ed7d87ef2deb","repo":"xai-org/grok-build","slug":"agent-runtime-worker-join-e","errorCode":null,"errorMessage":"agent runtime worker join: {e}","messagePattern":"agent runtime worker join: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/codegen/xai-grok-pager/src/acp/spawn.rs","lineNumber":270,"sourceCode":"}\n\n/// Spawn an agent in a dedicated thread with direct RPC dispatch.\n///\n/// The agent runs on a single-threaded tokio LocalSet runtime.\n/// RPC requests go directly to the agent via Rc, bypassing simplex pipes.\nasync fn spawn_agent_thread_direct(\n    spawn_agent: Box<dyn FnOnce(AcpClientTx) -> Result<Rc<MvpAgent>> + Send + 'static>,\n    channel: AcpAgentChannel,\n    cancel: CancellationToken,\n    skills_paths: Vec<String>,\n) -> Result<thread::JoinHandle<Result<()>>> {\n    // Off the UI worker: failure must fail spawn, not start ACP.\n    let rt = tokio::task::spawn_blocking(|| {\n        let mut builder = tokio::runtime::Builder::new_current_thread();\n        xai_tty_utils::runtime::build_with_blocking_pool(builder.enable_all())\n    })\n    .await\n    .map_err(|e| anyhow::anyhow!(\"agent runtime worker join: {e}\"))?\n    .map_err(|e| {\n        tracing::error!(error = %e, \"failed to start agent runtime\");\n        anyhow::anyhow!(\"failed to start agent runtime: {e}\")\n    })?;\n    Ok(thread::Builder::new()\n        .name(\"acp-agent-worker\".into())\n        .spawn(move || -> Result<()> {\n            let local = tokio::task::LocalSet::new();\n            local.block_on(&rt, async move {\n                let client_tx = channel.tx.clone();\n                let agent_rc = spawn_agent(client_tx)?;\n\n                // Direct dispatch: RPC requests go straight to the agent\n                let gw_rx =\n                    AcpGatewayReceiver::new(channel.rx, agent_rc.clone()).with_tracing(true);\n                tokio::task::spawn_local(gw_rx.run());\n\n                let _skills_watcher = {","sourceCodeStart":252,"sourceCodeEnd":288,"githubUrl":"https://github.com/xai-org/grok-build/blob/bc7f02eddd3d84085849dc19ed216f11c23b0571/crates/codegen/xai-grok-pager/src/acp/spawn.rs#L252-L288","documentation":"The spawn_blocking task that builds the dedicated tokio current-thread runtime for the ACP agent worker was joined with a JoinError — the blocking task panicked or was cancelled before producing a runtime. spawn_agent_thread_direct maps that to this error so spawn fails cleanly instead of starting ACP in a broken state.","triggerScenarios":"Calling spawn_agent_thread_direct (via spawn_grok_shell) when the spawn_blocking closure panics (e.g. build_with_blocking_pool fails/panics) or the blocking-pool task is cancelled.","commonSituations":"Runtime construction panicking under resource exhaustion (thread limits, memory), cancellation during shutdown racing spawn, bugs in blocking-pool configuration.","solutions":["Inspect logs for a panic in the runtime-builder task (JoinError payload)","Check thread/memory limits (ulimit -u, cgroup limits) that could kill worker threads","Retry spawn; if it recurs during shutdown, avoid racing spawn_agent_thread_direct with process teardown","Report/investigate build_with_blocking_pool panics in xai_tty_utils"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"match spawn_grok_shell(opts).await {\n    Err(e) if e.to_string().contains(\"agent runtime worker join\") => {\n        // JoinError: panic or cancellation; check panic payload in logs, retry once\n        retry_spawn(opts, 1).await\n    }\n    r => r,\n}","preventionTips":["Avoid racing spawn with process shutdown/cancellation","Monitor system thread limits before launching many agents","Capture JoinError panic payloads in logs for diagnosis","Keep xai_tty_utils runtime builder configuration valid and tested"],"tags":["tokio","runtime","spawn","panic"],"backgroundTag":"task-join-failed","analyzedSha":"bc7f02eddd3d84085849dc19ed216f11c23b0571","analyzedAt":"2026-08-31T04:59:42.031Z","schemaVersion":2},"datasetVersion":"2026-08-31T09:17:48.483Z"}