zellij-org/zellij · critical

failed to build forward-timeout runtime

Error message

failed to build forward-timeout runtime

What it means

Lazily builds the client's global forward-timeout executor: a one-thread tokio runtime with only the time driver enabled, held in a OnceLock and driven by a dedicated thread parked on block_on(pending()). Runtime construction is expected to succeed; it realistically fails only when the time driver cannot allocate its resources (fd exhaustion, EMFILE) or memory allocation fails. There is no fallback executor, so the first failing build panics the thread that scheduled the timeout.

Source

Thrown at zellij-client/src/stdin_ansi_parser.rs:903

// Forward-slot timeout infrastructure
// =====================================================================

use std::sync::{Arc, Mutex, OnceLock};

/// Dedicated, lazily-initialised runtime for driving forward-slot
/// timeouts. A single current-thread executor runs on its own OS
/// thread; timer tasks are `spawn`-ed onto it from the synchronous
/// `ClientInstruction::ForwardQueryToHost` handler. One-thread model
/// because timer tasks do no CPU work — they just sleep and perform a
/// millisecond-scale mutex check on wake-up.
static FORWARD_TIMEOUT_RUNTIME: OnceLock<Arc<tokio::runtime::Runtime>> = OnceLock::new();

pub fn forward_timeout_runtime() -> &'static Arc<tokio::runtime::Runtime> {
    FORWARD_TIMEOUT_RUNTIME.get_or_init(|| {
        let rt = tokio::runtime::Builder::new_current_thread()
            .enable_time()
            .build()
            .expect("failed to build forward-timeout runtime");
        let rt = Arc::new(rt);
        let rt_for_driver = rt.clone();
        // `block_on(pending())` keeps the executor loop alive forever
        // on this thread; spawned timer tasks are polled as they
        // become ready (on spawn, on wake from the time driver).
        std::thread::Builder::new()
            .name("zellij-client-forward-timeout".into())
            .spawn(move || {
                rt_for_driver.block_on(std::future::pending::<()>());
            })
            .expect("failed to spawn forward-timeout driver thread");
        rt
    })
}

/// Spawn a timer task that closes a forward slot after `deadline` and
/// invokes `on_timeout(token, reply_bytes)` with whatever the slot
/// accumulated. Token-guard idempotent: if the barrier (or a

View on GitHub (pinned to 98a0837077)

Solutions

1. Raise the fd limit before launching (ulimit -n 4096, or LimitNOFILE in systemd) and retry
2. Count open fds (`ls /proc/$(pidof zellij)/fd | wc -l`) to find leaks
3. Restart the client; the runtime is built once per process
4. If memory pressure is the cause, free memory or raise limits and retry

Example fix

// before
let rt = tokio::runtime::Builder::new_current_thread()
    .enable_time()
    .build()
    .expect("failed to build forward-timeout runtime");

// after - fail with context instead of a bare panic
let rt = tokio::runtime::Builder::new_current_thread()
    .enable_time()
    .build()
    .context("forward-timeout runtime: fd or memory exhaustion?")
    .fatal();

fn fd_headroom(min_free: usize) -> bool {
    match std::fs::read_dir("/proc/self/fd") {
        Ok(entries) => entries.count() + min_free < 1024, // compare against your RLIMIT_NOFILE
        Err(_) => true, // cannot tell; assume ok
    }
}

match tokio::runtime::Builder::new_current_thread().enable_time().build() {
    Ok(rt) => rt,
    Err(e) => {
        log::error!("tokio time runtime failed ({e}); falling back to std thread timer");
        // fallback: std::thread + mpsc with recv_timeout implements the same one-shot timeout
        fallback_std_timer()
    }
}

When it happens

Trigger: First call to forward_timeout_runtime() - scheduling a forward/query timeout - when the process is out of fds (RLIMIT_NOFILE) or under severe memory pressure.

Common situations: Long-lived clients on hosts with low `ulimit -n`; containers with tiny fd ceilings; fd leaks elsewhere in the process pushing it over the limit.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Failed to create tokio runtime
• failed to spawn forward-timeout driver thread
• Could not find editor pane to replace - is no pane focused?
• Could not find editor pane to replace
• failed to find active pane id for client {client_id}

AI-assisted analysis of zellij-org/zellij@98a0837077 (2026-08-16). Data as JSON: /api/errors/6f4f7ac5c06dac54. Report an issue: GitHub.