vllm-project/vllm · critical

failed to build vLLM ZMQ runtime

Error message

failed to build vLLM ZMQ runtime

What it means

This is a panic (`.expect`) in build_zmq_runtime when Tokio fails to construct the multi-threaded runtime dedicated to ZMQ tasks. `Builder::build()` fails essentially only when worker threads cannot be spawned (resource exhaustion) — this is not a recoverable Result-based error but a process abort at client startup.

Source

Thrown at rust/src/engine-core-client/src/runtime.rs:65

/// small, and multiple engines share the same ZMQ socket. Therefore, based on
/// benchmarks, a default value of 4 is generally sufficient.
const DEFAULT_ZMQ_WORKER_THREADS: usize = 4;

static ZMQ_RUNTIME_SEQUENCE: OnceLock<AtomicUsize> = OnceLock::new();

/// Build a Tokio runtime for ZMQ tasks. Multiple calls to this function will
/// return multiple runtimes with distinct thread name suffixes.
pub(crate) fn build_zmq_runtime() -> BackgroundShutdownRuntime {
    let sequence = ZMQ_RUNTIME_SEQUENCE
        .get_or_init(|| AtomicUsize::new(0))
        .fetch_add(1, Ordering::Relaxed);

    tokio::runtime::Builder::new_multi_thread()
        .worker_threads(zmq_worker_threads())
        .thread_name_fn(move || format!("vllm-zmq-{sequence}"))
        .enable_all()
        .build()
        .expect("failed to build vLLM ZMQ runtime")
        .into()
}

/// Get the number of worker threads to use for the ZMQ runtime. If env var
/// `VLLM_RS_ZMQ_WORKER_THREADS` is set and a valid positive integer, it will be used.
/// Otherwise, the default value of `DEFAULT_ZMQ_WORKER_THREADS` will be used.
fn zmq_worker_threads() -> usize {
    std::env::var(ZMQ_WORKER_THREADS_ENV)
        .ok()
        .and_then(|value| value.parse::<usize>().ok())
        .filter(|value| *value > 0)
        .unwrap_or(DEFAULT_ZMQ_WORKER_THREADS)
}

View on GitHub (pinned to c794754062)

Solutions

1. Raise thread limits: ulimit -u / cgroup pids.max / container task limits
2. Reduce the number of concurrently created EngineCoreClient instances in the process
3. Set VLLM_RS_ZMQ_WORKER_THREADS to a small positive value to lower per-runtime thread count
4. Free system memory — thread stack allocation failure also surfaces here

Example fix

# before (cgroup/container limit)
pids.max = 64

# after
pids.max = 512

# or cap ZMQ threads
export VLLM_RS_ZMQ_WORKER_THREADS=1

// Pre-flight: can we spawn the threads this runtime needs?
let want = std::env::var("VLLM_RS_ZMQ_WORKER_THREADS").ok().and_then(|v| v.parse::<usize>().ok()).filter(|v| *v > 0).unwrap_or(2);
let avail = thread::available_parallelism().map(|n| n.get()).unwrap_or(1);
if want > avail { eprintln!("raise thread limits or lower VLLM_RS_ZMQ_WORKER_THREADS"); }

// This is a panic via .expect, not a Result — catch_unwind only contains the blast radius
let rt = std::panic::catch_unwind(build_zmq_runtime_extern)
    .unwrap_or_else(|_| panic::resume_unwind(Box::new("ZMQ runtime spawn failed: check ulimit -u / pids.max")));

When it happens

Trigger: Creating an EngineCoreClient (each builds its own ZMQ runtime) on a system that cannot spawn more threads: thread/memory limits (ulimit, cgroups), or an absurd value forced via VLLM_RS_ZMQ_WORKER_THREADS.

Common situations: Containers with low nproc/task limits creating many engine clients; RLIMIT_NPROC hit in CI sandboxes; heavy test suites instantiating many clients in one process.

Related errors

• ZMQ runtime task failed
• failed to build request runtime
• tool response messages require a tool_call_id; use ChatMessa
• engine control channel closed unexpectedly: {message}
• engine-core output dispatcher closed: {message}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/0f3dc842d89b1bbc. Report an issue: GitHub.