{"record":{"id":"fd6489b389ef87ff","repo":"vllm-project/vllm","slug":"endpoint-not-ready-after-0-s-1","errorCode":null,"errorMessage":"Endpoint not ready after {0}s: {1}","messagePattern":"Endpoint not ready after (.+?)s: (.+?)","errorType":"exception","errorClass":"BenchError","httpStatus":null,"severity":"error","filePath":"rust/src/bench/src/error.rs","lineNumber":27,"sourceCode":"    Http(#[from] reqwest::Error),\n\n    #[error(\"JSON error: {0}\")]\n    Json(#[from] serde_json::Error),\n\n    #[error(\"Tokenizer error: {0}\")]\n    Tokenizer(String),\n\n    /// The server's /tokenize//detokenize endpoint is not usable (4xx status:\n    /// not exposed, or rejected by a gateway such as LLM-d/EPP that returns\n    /// 400 instead of 404). Callers treat this as \"skip verification\", unlike\n    /// `Tokenizer` errors which are genuine failures.\n    #[error(\"tokenize endpoint unavailable: {0}\")]\n    TokenizeUnavailable(String),\n\n    #[error(\"Configuration error: {0}\")]\n    Config(String),\n\n    #[error(\"Endpoint not ready after {0}s: {1}\")]\n    EndpointTimeout(u64, String),\n\n    #[error(\"Backend error: {0}\")]\n    Backend(String),\n\n    #[error(\"IO error: {0}\")]\n    Io(#[from] std::io::Error),\n}\n\npub type Result<T> = std::result::Result<T, BenchError>;\n","sourceCodeStart":9,"sourceCodeEnd":38,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/rust/src/bench/src/error.rs#L9-L38","documentation":"During deferred NIXL remote-state setup, each rank all-gathers metadata about its expert weight tensors. Every rank must describe exactly the same set of tensor keys (per MoE layer). Rank {peer} advertised a different key set than the local rank, so descriptor exchange cannot proceed safely.","triggerScenarios":"Elastic EP ranks loading different MoE models or different numbers of MoE layers (e.g. one rank configured with a different model path or PP stage split); a rank that skipped loading a layer's expert weights before set_transfer_context ran.","commonSituations":"Heterogeneous or partially-updated model rollout across elastic workers; PP pipeline splits that differ per rank; bug where some ranks have empty expert weights for a layer.","solutions":["Verify all elastic EP ranks load the same model and MoE architecture (same number of MoE layers and expert tensor keys)","Ensure every rank finishes load_model before the first set_transfer_context / reconfiguration collective","Diff the local=[...] and peer=[...] key lists in the message to identify which layer/tensor is missing or extra"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"# before the collective: assert every rank has identical expert tensor keys\nlocal_keys = {name for name, _ in named_expert_tensors(model)}\nexpected = all_gather_object_once(local_keys)\nassert all(k == local_keys for k in expected), f'key mismatch: {expected}'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Launch all elastic ranks from the same model artifact and config","Complete load_model on every rank before triggering reconfiguration","Log per-rank expert tensor key sets during bring-up to catch divergence early"],"tags":["nixl","eplb","elastic-ep","metadata-mismatch","vllm"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}