vllm-project/vllm · critical · Error

engine-core output dispatcher closed: {message}

Error message

engine-core output dispatcher closed: {message}

What it means

EngineCoreError::DispatcherClosed means the internal task that demultiplexes engine-core outputs (the dispatcher) terminated while the client was still using it. Every request stream ultimately depends on this dispatcher, so its closure indicates the engine-core output channel (ZMQ) broke or the dispatcher task panicked.

Source

Thrown at rust/src/engine-core-client/src/error.rs:78

    #[error("external coordinator mode is not implemented yet")]
    UnsupportedExternalCoordinator,
    #[error("unsupported field `{field}` in {context}")]
    UnsupportedField {
        context: &'static str,
        field: &'static str,
    },
    #[error("engine control channel closed unexpectedly: {message}")]
    ControlClosed { message: String },
    #[error("request `{request_id}` is already in flight")]
    DuplicateRequestId { request_id: String },
    #[error(
        "data parallel rank {rank} is not connected to this frontend; connected ranks: {connected_ranks:?}"
    )]
    InvalidDataParallelRank {
        rank: u32,
        connected_ranks: Vec<u32>,
    },
    #[error("engine-core output dispatcher closed: {message}")]
    DispatcherClosed { message: String },
    #[error("engine-core client is closed: {message}")]
    ClientClosed { message: String },
    #[error("request output stream for `{request_id}` closed unexpectedly")]
    RequestStreamClosed { request_id: String },
    #[error("utility call `{method}` failed (call_id={call_id}): {message}")]
    UtilityCallFailed {
        method: String,
        call_id: UtilityCallId,
        message: String,
    },
    #[error("utility call `{method}` returned an invalid result (call_id={call_id}): {message}")]
    UtilityResultDecode {
        method: String,
        call_id: UtilityCallId,
        message: String,
    },
    #[error("utility call `{method}` closed unexpectedly (call_id={call_id})")]

View on GitHub (pinned to c794754062)

Solutions

1. Inspect engine-core stderr/logs for the crash that killed the output channel
2. Address the root cause (memory limits, model config, dispatcher decode bug)
3. Restart the engine-core and rebuild the client to get a fresh dispatcher

pub fn is_dispatcher_closed(e: &vllm_engine_core_client::Error) -> bool {
    matches!(e, vllm_engine_core_client::Error::DispatcherClosed { .. })
}

match stream.next().await {
    Some(Err(e @ vllm_engine_core_client::Error::DispatcherClosed { .. })) => {
        tracing::error!("output dispatcher died: {e}");
        // all in-flight requests are dead: surface to callers and restart engine
        Err(e.into())
    }
    other => other.transpose().map_err(Into::into),
}

When it happens

Trigger: Any client operation requiring engine-core output (generate stream, utility calls) after the engine-core process died, the output ZMQ socket closed, or the dispatcher task panicked while decoding a frame.

Common situations: Engine-core crash (OOM, panic in scheduler); ZMQ endpoint mismatch or socket exhaustion; abrupt engine shutdown while requests are streaming.

Related errors

• engine control channel closed unexpectedly: {message}
• unexpected output on main dispatcher path: {message}
• data parallel rank {rank} is not connected to this frontend;
• request output stream for `{request_id}` closed unexpectedly
• utility call `{method}` failed (call_id={call_id}): {message

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/e384a6760a189f74. Report an issue: GitHub.