xai-org/grok-build · error

failed to start agent runtime: {e}

Error message

failed to start agent runtime: {e}

What it means

The blocking task ran but xai_tty_utils::runtime::build_with_blocking_pool returned an Err when constructing the agent's dedicated tokio runtime. The error is logged via tracing and then wrapped as 'failed to start agent runtime: {e}'.

Source

Thrown at crates/codegen/xai-grok-pager/src/acp/spawn.rs:273

///
/// The agent runs on a single-threaded tokio LocalSet runtime.
/// RPC requests go directly to the agent via Rc, bypassing simplex pipes.
async fn spawn_agent_thread_direct(
    spawn_agent: Box<dyn FnOnce(AcpClientTx) -> Result<Rc<MvpAgent>> + Send + 'static>,
    channel: AcpAgentChannel,
    cancel: CancellationToken,
    skills_paths: Vec<String>,
) -> Result<thread::JoinHandle<Result<()>>> {
    // Off the UI worker: failure must fail spawn, not start ACP.
    let rt = tokio::task::spawn_blocking(|| {
        let mut builder = tokio::runtime::Builder::new_current_thread();
        xai_tty_utils::runtime::build_with_blocking_pool(builder.enable_all())
    })
    .await
    .map_err(|e| anyhow::anyhow!("agent runtime worker join: {e}"))?
    .map_err(|e| {
        tracing::error!(error = %e, "failed to start agent runtime");
        anyhow::anyhow!("failed to start agent runtime: {e}")
    })?;
    Ok(thread::Builder::new()
        .name("acp-agent-worker".into())
        .spawn(move || -> Result<()> {
            let local = tokio::task::LocalSet::new();
            local.block_on(&rt, async move {
                let client_tx = channel.tx.clone();
                let agent_rc = spawn_agent(client_tx)?;

                // Direct dispatch: RPC requests go straight to the agent
                let gw_rx =
                    AcpGatewayReceiver::new(channel.rx, agent_rc.clone()).with_tracing(true);
                tokio::task::spawn_local(gw_rx.run());

                let _skills_watcher = {
                    let cwd = std::env::current_dir().unwrap_or_default();
                    let workspace_user_dir =
                        xai_grok_agent::prompt::workspace_user::optional_workspace_user_dir();

View on GitHub (pinned to bc7f02eddd)

Solutions

1. Read the underlying {e} from logs (tracing 'failed to start agent runtime')
2. Raise ulimit -n (file descriptors) and thread limits, or container resource caps
3. Retry spawn after resources are freed; avoid spawning many agent workers concurrently
4. Pin/verify the xai_tty_utils runtime builder configuration

Example fix

// before
let rt = build_with_blocking_pool(builder.enable_all())?;
// after
let rt = build_with_blocking_pool(builder.enable_all())
    .map_err(|e| { tracing::error!(error = %e, "failed to start agent runtime"); e })?;

// preflight: can the process create a thread and open an epoll fd?
fn runtime_prereqs_ok() -> bool {
    std::thread::Builder::new().stack_size(64 * 1024).spawn(|_| {}).map(|h| h.join().ok()).is_ok()
}

match spawn_grok_shell(opts).await {
    Err(e) if e.to_string().contains("failed to start agent runtime") => {
        eprintln!("runtime build failed: {e:#}"); // check fd/thread limits, then retry
        retry_spawn(opts, 1).await
    }
    r => r,
}

When it happens

Trigger: Calling spawn_agent_thread_direct when tokio runtime Builder (new_current_thread + enable_all + blocking pool) fails to build, typically due to OS resource limits (threads, file descriptors, io driver setup).

Common situations: Hitting thread/process limits on constrained machines, invalid runtime configuration in the blocking pool builder, low fd limits preventing epoll driver creation, containers with restrictive limits.

Related errors

• agent runtime worker join: {e}
• git discover task failed: {error}
• blocking pool pre-warm stalled after {started} of {n} thread
• Connection cancelled
• PTY write channel closed

AI-assisted analysis of xai-org/grok-build@bc7f02eddd (2026-08-31). Data as JSON: /api/errors/b67824dd2bf6f6dd. Report an issue: GitHub.