{"record":{"id":"b67824dd2bf6f6dd","repo":"xai-org/grok-build","slug":"failed-to-start-agent-runtime-e","errorCode":null,"errorMessage":"failed to start agent runtime: {e}","messagePattern":"failed to start agent runtime: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/codegen/xai-grok-pager/src/acp/spawn.rs","lineNumber":273,"sourceCode":"///\n/// The agent runs on a single-threaded tokio LocalSet runtime.\n/// RPC requests go directly to the agent via Rc, bypassing simplex pipes.\nasync fn spawn_agent_thread_direct(\n    spawn_agent: Box<dyn FnOnce(AcpClientTx) -> Result<Rc<MvpAgent>> + Send + 'static>,\n    channel: AcpAgentChannel,\n    cancel: CancellationToken,\n    skills_paths: Vec<String>,\n) -> Result<thread::JoinHandle<Result<()>>> {\n    // Off the UI worker: failure must fail spawn, not start ACP.\n    let rt = tokio::task::spawn_blocking(|| {\n        let mut builder = tokio::runtime::Builder::new_current_thread();\n        xai_tty_utils::runtime::build_with_blocking_pool(builder.enable_all())\n    })\n    .await\n    .map_err(|e| anyhow::anyhow!(\"agent runtime worker join: {e}\"))?\n    .map_err(|e| {\n        tracing::error!(error = %e, \"failed to start agent runtime\");\n        anyhow::anyhow!(\"failed to start agent runtime: {e}\")\n    })?;\n    Ok(thread::Builder::new()\n        .name(\"acp-agent-worker\".into())\n        .spawn(move || -> Result<()> {\n            let local = tokio::task::LocalSet::new();\n            local.block_on(&rt, async move {\n                let client_tx = channel.tx.clone();\n                let agent_rc = spawn_agent(client_tx)?;\n\n                // Direct dispatch: RPC requests go straight to the agent\n                let gw_rx =\n                    AcpGatewayReceiver::new(channel.rx, agent_rc.clone()).with_tracing(true);\n                tokio::task::spawn_local(gw_rx.run());\n\n                let _skills_watcher = {\n                    let cwd = std::env::current_dir().unwrap_or_default();\n                    let workspace_user_dir =\n                        xai_grok_agent::prompt::workspace_user::optional_workspace_user_dir();","sourceCodeStart":255,"sourceCodeEnd":291,"githubUrl":"https://github.com/xai-org/grok-build/blob/bc7f02eddd3d84085849dc19ed216f11c23b0571/crates/codegen/xai-grok-pager/src/acp/spawn.rs#L255-L291","documentation":"The blocking task ran but xai_tty_utils::runtime::build_with_blocking_pool returned an Err when constructing the agent's dedicated tokio runtime. The error is logged via tracing and then wrapped as 'failed to start agent runtime: {e}'.","triggerScenarios":"Calling spawn_agent_thread_direct when tokio runtime Builder (new_current_thread + enable_all + blocking pool) fails to build, typically due to OS resource limits (threads, file descriptors, io driver setup).","commonSituations":"Hitting thread/process limits on constrained machines, invalid runtime configuration in the blocking pool builder, low fd limits preventing epoll driver creation, containers with restrictive limits.","solutions":["Read the underlying {e} from logs (tracing 'failed to start agent runtime')","Raise ulimit -n (file descriptors) and thread limits, or container resource caps","Retry spawn after resources are freed; avoid spawning many agent workers concurrently","Pin/verify the xai_tty_utils runtime builder configuration"],"exampleFix":"// before\nlet rt = build_with_blocking_pool(builder.enable_all())?;\n// after\nlet rt = build_with_blocking_pool(builder.enable_all())\n    .map_err(|e| { tracing::error!(error = %e, \"failed to start agent runtime\"); e })?;","handlingStrategy":"retry","validationCode":"// preflight: can the process create a thread and open an epoll fd?\nfn runtime_prereqs_ok() -> bool {\n    std::thread::Builder::new().stack_size(64 * 1024).spawn(|_| {}).map(|h| h.join().ok()).is_ok()\n}","typeGuard":null,"tryCatchPattern":"match spawn_grok_shell(opts).await {\n    Err(e) if e.to_string().contains(\"failed to start agent runtime\") => {\n        eprintln!(\"runtime build failed: {e:#}\"); // check fd/thread limits, then retry\n        retry_spawn(opts, 1).await\n    }\n    r => r,\n}","preventionTips":["Raise ulimit -n and thread limits on hosts running many agents","Avoid building runtimes inside tight containers with low resource caps","Check the tracing error log for the underlying builder failure","Refrain from spawning multiple agent workers simultaneously during startup"],"tags":["tokio","runtime","resources","spawn"],"backgroundTag":"runtime-start-failed","analyzedSha":"bc7f02eddd3d84085849dc19ed216f11c23b0571","analyzedAt":"2026-08-31T04:59:42.031Z","schemaVersion":2},"datasetVersion":"2026-08-31T09:17:48.483Z"}