vllm-project/vllm · error · BenchError

Endpoint not ready after {0}s: {1}

Error message

Endpoint not ready after {0}s: {1}

What it means

During deferred NIXL remote-state setup, each rank all-gathers metadata about its expert weight tensors. Every rank must describe exactly the same set of tensor keys (per MoE layer). Rank {peer} advertised a different key set than the local rank, so descriptor exchange cannot proceed safely.

Source

Thrown at rust/src/bench/src/error.rs:27

    Http(#[from] reqwest::Error),

    #[error("JSON error: {0}")]
    Json(#[from] serde_json::Error),

    #[error("Tokenizer error: {0}")]
    Tokenizer(String),

    /// The server's /tokenize//detokenize endpoint is not usable (4xx status:
    /// not exposed, or rejected by a gateway such as LLM-d/EPP that returns
    /// 400 instead of 404). Callers treat this as "skip verification", unlike
    /// `Tokenizer` errors which are genuine failures.
    #[error("tokenize endpoint unavailable: {0}")]
    TokenizeUnavailable(String),

    #[error("Configuration error: {0}")]
    Config(String),

    #[error("Endpoint not ready after {0}s: {1}")]
    EndpointTimeout(u64, String),

    #[error("Backend error: {0}")]
    Backend(String),

    #[error("IO error: {0}")]
    Io(#[from] std::io::Error),
}

pub type Result<T> = std::result::Result<T, BenchError>;

View on GitHub (pinned to c794754062)

Solutions

1. Verify all elastic EP ranks load the same model and MoE architecture (same number of MoE layers and expert tensor keys)
2. Ensure every rank finishes load_model before the first set_transfer_context / reconfiguration collective
3. Diff the local=[...] and peer=[...] key lists in the message to identify which layer/tensor is missing or extra

# before the collective: assert every rank has identical expert tensor keys
local_keys = {name for name, _ in named_expert_tensors(model)}
expected = all_gather_object_once(local_keys)
assert all(k == local_keys for k in expected), f'key mismatch: {expected}'

When it happens

Trigger: Elastic EP ranks loading different MoE models or different numbers of MoE layers (e.g. one rank configured with a different model path or PP stage split); a rank that skipped loading a layer's expert weights before set_transfer_context ran.

Common situations: Heterogeneous or partially-updated model rollout across elastic workers; PP pipeline splits that differ per rank; bug where some ranks have empty expert weights for a layer.

Related errors

• Backend error: {0}
• Configuration error: {0}
• IO error: {0}
• chat template error: {0}
• multimodal input is not supported by this chat renderer

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/fd6489b389ef87ff. Report an issue: GitHub.