What causes "engine-core error" in vllm-project/vllm?

Match or print the inner EngineCoreClient error (`err.to_string()` / downcast) to identify the actual variant

vllm-project/vllm · error · vllm_llm::Error

engine-core error

Error message

engine-core error

What it means

llm::Error::EngineCoreClient is a transparent wrapper (#[from]) around vllm_engine_core_client::Error, displayed simply as "engine-core error". Any failure raised by the underlying client layer — closed channels, duplicate request IDs, utility call failures, engine crashes — surfaces through this variant in the public LLM API.

Source

Thrown at rust/src/llm/src/error.rs:13

// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: Copyright contributors to the vLLM project

use thiserror::Error;

pub type Result<T> = std::result::Result<T, Error>;

/// Public error type for the Rust `llm` facade.
#[derive(Debug, Error)]
pub enum Error {
    #[error("generate request `{request_id}` has an empty prompt_token_ids")]
    EmptyPromptTokenIds { request_id: String },
    #[error("engine-core error")]
    EngineCoreClient(#[from] vllm_engine_core_client::Error),
}

View on GitHub (pinned to c794754062)

Solutions

Match or print the inner EngineCoreClient error (`err.to_string()` / downcast) to identify the actual variant
Apply the fix corresponding to the inner error (see ControlClosed, DispatcherClosed, DuplicateRequestId, etc.)
Enable debug logging on vllm_engine_core_client for the full error chain

Example fix

// before
match llm.generate(req).await { Err(e) => log::error!("failed: {e}"), .. }

// after
match llm.generate(req).await {
    Err(e) => match e {
        llm::Error::EngineCoreClient(inner) => log::error!("engine-core: {inner:?}"),
        other => log::error!("other: {other}"),
    },
    ..
}

Defensive patterns

Strategy: try-catch

Type guard

fn is_engine_core(e: &vllm_llm::Error) -> bool {
    matches!(e, vllm_llm::Error::EngineCoreClient(_))
}

Try / catch

match llm.generate(req).await {
    Err(vllm_llm::Error::EngineCoreClient(inner)) => {
        tracing::error!("engine-core failure: {inner}"); // inner Display names the real variant
        Err(vllm_llm::Error::EngineCoreClient(inner))
    }
    Err(e) => Err(e),
    Ok(v) => Ok(v),
}

Prevention

Always inspect the inner EngineCoreClient error instead of branching on the generic wrapper text
Log the full error chain ({e:#} with anyhow) to preserve the cause
Centralize the match on inner variants so retry/recovery policy lives in one place

When it happens

Trigger: Any LLM facade call (generate, shutdown, utilities) where the engine-core client returns an error: engine process death, channel closure, duplicate request_id, DP rank issues, or utility call failures.

Common situations: Users of the high-level Rust llm crate seeing a generic engine-core error; the specific cause is only visible by matching the inner error variant or its Display text.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/df30392425368dd0. Report an issue: GitHub.