{"record":{"id":"0f3dc842d89b1bbc","repo":"vllm-project/vllm","slug":"failed-to-build-vllm-zmq-runtime","errorCode":null,"errorMessage":"failed to build vLLM ZMQ runtime","messagePattern":"failed to build vLLM ZMQ runtime","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"rust/src/engine-core-client/src/runtime.rs","lineNumber":65,"sourceCode":"/// small, and multiple engines share the same ZMQ socket. Therefore, based on\n/// benchmarks, a default value of 4 is generally sufficient.\nconst DEFAULT_ZMQ_WORKER_THREADS: usize = 4;\n\nstatic ZMQ_RUNTIME_SEQUENCE: OnceLock<AtomicUsize> = OnceLock::new();\n\n/// Build a Tokio runtime for ZMQ tasks. Multiple calls to this function will\n/// return multiple runtimes with distinct thread name suffixes.\npub(crate) fn build_zmq_runtime() -> BackgroundShutdownRuntime {\n    let sequence = ZMQ_RUNTIME_SEQUENCE\n        .get_or_init(|| AtomicUsize::new(0))\n        .fetch_add(1, Ordering::Relaxed);\n\n    tokio::runtime::Builder::new_multi_thread()\n        .worker_threads(zmq_worker_threads())\n        .thread_name_fn(move || format!(\"vllm-zmq-{sequence}\"))\n        .enable_all()\n        .build()\n        .expect(\"failed to build vLLM ZMQ runtime\")\n        .into()\n}\n\n/// Get the number of worker threads to use for the ZMQ runtime. If env var\n/// `VLLM_RS_ZMQ_WORKER_THREADS` is set and a valid positive integer, it will be used.\n/// Otherwise, the default value of `DEFAULT_ZMQ_WORKER_THREADS` will be used.\nfn zmq_worker_threads() -> usize {\n    std::env::var(ZMQ_WORKER_THREADS_ENV)\n        .ok()\n        .and_then(|value| value.parse::<usize>().ok())\n        .filter(|value| *value > 0)\n        .unwrap_or(DEFAULT_ZMQ_WORKER_THREADS)\n}\n","sourceCodeStart":47,"sourceCodeEnd":79,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/rust/src/engine-core-client/src/runtime.rs#L47-L79","documentation":"This is a panic (`.expect`) in build_zmq_runtime when Tokio fails to construct the multi-threaded runtime dedicated to ZMQ tasks. `Builder::build()` fails essentially only when worker threads cannot be spawned (resource exhaustion) — this is not a recoverable Result-based error but a process abort at client startup.","triggerScenarios":"Creating an EngineCoreClient (each builds its own ZMQ runtime) on a system that cannot spawn more threads: thread/memory limits (ulimit, cgroups), or an absurd value forced via VLLM_RS_ZMQ_WORKER_THREADS.","commonSituations":"Containers with low nproc/task limits creating many engine clients; RLIMIT_NPROC hit in CI sandboxes; heavy test suites instantiating many clients in one process.","solutions":["Raise thread limits: ulimit -u / cgroup pids.max / container task limits","Reduce the number of concurrently created EngineCoreClient instances in the process","Set VLLM_RS_ZMQ_WORKER_THREADS to a small positive value to lower per-runtime thread count","Free system memory — thread stack allocation failure also surfaces here"],"exampleFix":"# before (cgroup/container limit)\npids.max = 64\n\n# after\npids.max = 512\n\n# or cap ZMQ threads\nexport VLLM_RS_ZMQ_WORKER_THREADS=1","handlingStrategy":"validation","validationCode":"// Pre-flight: can we spawn the threads this runtime needs?\nlet want = std::env::var(\"VLLM_RS_ZMQ_WORKER_THREADS\").ok().and_then(|v| v.parse::<usize>().ok()).filter(|v| *v > 0).unwrap_or(2);\nlet avail = thread::available_parallelism().map(|n| n.get()).unwrap_or(1);\nif want > avail { eprintln!(\"raise thread limits or lower VLLM_RS_ZMQ_WORKER_THREADS\"); }","typeGuard":null,"tryCatchPattern":"// This is a panic via .expect, not a Result — catch_unwind only contains the blast radius\nlet rt = std::panic::catch_unwind(build_zmq_runtime_extern)\n    .unwrap_or_else(|_| panic::resume_unwind(Box::new(\"ZMQ runtime spawn failed: check ulimit -u / pids.max\")));","preventionTips":["Set container/cgroup pid limits above total expected threads (engines x workers)","Keep VLLM_RS_ZMQ_WORKER_THREADS small when embedding many clients in one process","Monitor threads count and memory at startup in constrained environments"],"tags":["panic","tokio","resource-exhaustion","zmq","rust"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}