What causes "engine-core client is closed: {message}" in vllm-project/vllm?

Audit call sites that run concurrently with shutdown and gate them with a shutdown signal/ CancellationToken

vllm-project/vllm · error · Error

engine-core client is closed: {message}

Error message

engine-core client is closed: {message}

What it means

EngineCoreError::ClientClosed is returned when an operation is attempted on an EngineCoreClient that has already been shut down. The client keeps closed state after shutdown(), and any subsequent generate/abort/utility call hits this guard instead of touching dead sockets.

Source

Thrown at rust/src/engine-core-client/src/error.rs:80

    #[error("unsupported field `{field}` in {context}")]
    UnsupportedField {
        context: &'static str,
        field: &'static str,
    },
    #[error("engine control channel closed unexpectedly: {message}")]
    ControlClosed { message: String },
    #[error("request `{request_id}` is already in flight")]
    DuplicateRequestId { request_id: String },
    #[error(
        "data parallel rank {rank} is not connected to this frontend; connected ranks: {connected_ranks:?}"
    )]
    InvalidDataParallelRank {
        rank: u32,
        connected_ranks: Vec<u32>,
    },
    #[error("engine-core output dispatcher closed: {message}")]
    DispatcherClosed { message: String },
    #[error("engine-core client is closed: {message}")]
    ClientClosed { message: String },
    #[error("request output stream for `{request_id}` closed unexpectedly")]
    RequestStreamClosed { request_id: String },
    #[error("utility call `{method}` failed (call_id={call_id}): {message}")]
    UtilityCallFailed {
        method: String,
        call_id: UtilityCallId,
        message: String,
    },
    #[error("utility call `{method}` returned an invalid result (call_id={call_id}): {message}")]
    UtilityResultDecode {
        method: String,
        call_id: UtilityCallId,
        message: String,
    },
    #[error("utility call `{method}` closed unexpectedly (call_id={call_id})")]
    UtilityCallClosed { method: String, call_id: u64 },
    #[error("utility call `{method}` returned inconsistent results across engines: {values}")]

View on GitHub (pinned to c794754062)

Solutions

Audit call sites that run concurrently with shutdown and gate them with a shutdown signal/ CancellationToken
Create a new EngineCoreClient if you need to keep serving after a shutdown
Ensure background tasks (utility calls, stream consumers) are joined before calling shutdown()

Example fix

// before
let handle = tokio::spawn(client.clone().run_utility_loop());
client.shutdown().await; // background task still uses client

// after
let handle = tokio::spawn(client.clone().run_utility_loop());
shutdown_tx.send(()); 
handle.await;
client.shutdown().await;

Defensive patterns

Strategy: validation

Validate before calling

// Guard calls with a lifecycle flag
if client.is_closed() { return Err(anyhow::anyhow!("client already shut down")); }
client.generate(request_id, req).await

Type guard

pub fn is_client_closed(e: &vllm_engine_core_client::Error) -> bool {
    matches!(e, vllm_engine_core_client::Error::ClientClosed { .. })
}

Try / catch

match client.generate(/*..*/).await {
    Err(e @ vllm_engine_core_client::Error::ClientClosed { .. }) => {
        tracing::warn!("client closed before call; skipping: {e}");
        Ok(Default::default()) // or propagate as poisoned-state
    }
    other => other,
}

Prevention

Broadcast a shutdown token; all tasks must observe it and stop calling the client before shutdown() runs
Join background utility/stream tasks before initiating client shutdown
Never reuse a client after shutdown — create a fresh one for a new engine session

When it happens

Trigger: Calling any API on the client after shutdown() has completed; sharing an Arc<EngineCoreClient> across tasks where one task shuts down while another still issues requests; using a client dropped from a connection pool and re-initialized to closed state.

Common situations: Task-join ordering bugs during graceful shutdown; a background metrics/utility task outliving the main serving loop; re-using a client after an error path already triggered shutdown.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/482dc61f5cd90073. Report an issue: GitHub.