{"record":{"id":"5f7059da0a6340d6","repo":"vllm-project/vllm","slug":"backend-error-0","errorCode":null,"errorMessage":"Backend error: {0}","messagePattern":"Backend error: (.+?)","errorType":"exception","errorClass":"BenchError","httpStatus":null,"severity":"error","filePath":"rust/src/bench/src/error.rs","lineNumber":30,"sourceCode":"    Json(#[from] serde_json::Error),\n\n    #[error(\"Tokenizer error: {0}\")]\n    Tokenizer(String),\n\n    /// The server's /tokenize//detokenize endpoint is not usable (4xx status:\n    /// not exposed, or rejected by a gateway such as LLM-d/EPP that returns\n    /// 400 instead of 404). Callers treat this as \"skip verification\", unlike\n    /// `Tokenizer` errors which are genuine failures.\n    #[error(\"tokenize endpoint unavailable: {0}\")]\n    TokenizeUnavailable(String),\n\n    #[error(\"Configuration error: {0}\")]\n    Config(String),\n\n    #[error(\"Endpoint not ready after {0}s: {1}\")]\n    EndpointTimeout(u64, String),\n\n    #[error(\"Backend error: {0}\")]\n    Backend(String),\n\n    #[error(\"IO error: {0}\")]\n    Io(#[from] std::io::Error),\n}\n\npub type Result<T> = std::result::Result<T, BenchError>;\n","sourceCodeStart":12,"sourceCodeEnd":38,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/rust/src/bench/src/error.rs#L12-L38","documentation":"NIXL EPLB remote-state setup requires every rank's per-expert tensor byte strides (nbytes_per_expert) to match for each tensor key. For key {key}, rank {peer} reported {peer_stride} bytes per expert while the local rank has {local_stride}. Different strides mean incompatible buffer layouts and transfers would corrupt data.","triggerScenarios":"Ranks with different dtypes (e.g. one rank quantized FP8, another BF16), different expert world sizes producing different padded buffer sizes, or different quantization formats for the same layer's expert weights.","commonSituations":"Mixed-precision deployments where one worker was launched with a different quant config; heterogeneous GPUs forcing different dtype strategies; partial rollout of a quantization change during elastic scaling.","solutions":["Make dtype and quantization config identical across all elastic EP ranks (check the {key} tensor's dtype on both ranks)","Ensure the same EP world-size parameters so per-expert buffer padding matches","Restart the elastic group from a consistent configuration rather than joining ranks with mismatched configs"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"# check per-expert byte strides match across ranks before set_transfer_context\nlocal_stride = {k: nbytes_per_expert(t) for k, t in expert_tensors.items()}\npeers = all_gather_object_once(local_stride)\nassert all(p == local_stride for p in peers), f'stride mismatch: {peers}'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep dtype and quantization config identical across all elastic ranks","Never join ranks with different EP world-size parameters to a live group","Include dtype/stride in the elastic join handshake validation"],"tags":["nixl","eplb","elastic-ep","dtype-mismatch","vllm"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}