vllm-project/vllm · error · BenchError

Backend error: {0}

Error message

Backend error: {0}

What it means

NIXL EPLB remote-state setup requires every rank's per-expert tensor byte strides (nbytes_per_expert) to match for each tensor key. For key {key}, rank {peer} reported {peer_stride} bytes per expert while the local rank has {local_stride}. Different strides mean incompatible buffer layouts and transfers would corrupt data.

Source

Thrown at rust/src/bench/src/error.rs:30

    Json(#[from] serde_json::Error),

    #[error("Tokenizer error: {0}")]
    Tokenizer(String),

    /// The server's /tokenize//detokenize endpoint is not usable (4xx status:
    /// not exposed, or rejected by a gateway such as LLM-d/EPP that returns
    /// 400 instead of 404). Callers treat this as "skip verification", unlike
    /// `Tokenizer` errors which are genuine failures.
    #[error("tokenize endpoint unavailable: {0}")]
    TokenizeUnavailable(String),

    #[error("Configuration error: {0}")]
    Config(String),

    #[error("Endpoint not ready after {0}s: {1}")]
    EndpointTimeout(u64, String),

    #[error("Backend error: {0}")]
    Backend(String),

    #[error("IO error: {0}")]
    Io(#[from] std::io::Error),
}

pub type Result<T> = std::result::Result<T, BenchError>;

View on GitHub (pinned to c794754062)

Solutions

1. Make dtype and quantization config identical across all elastic EP ranks (check the {key} tensor's dtype on both ranks)
2. Ensure the same EP world-size parameters so per-expert buffer padding matches
3. Restart the elastic group from a consistent configuration rather than joining ranks with mismatched configs

# check per-expert byte strides match across ranks before set_transfer_context
local_stride = {k: nbytes_per_expert(t) for k, t in expert_tensors.items()}
peers = all_gather_object_once(local_stride)
assert all(p == local_stride for p in peers), f'stride mismatch: {peers}'

When it happens

Trigger: Ranks with different dtypes (e.g. one rank quantized FP8, another BF16), different expert world sizes producing different padded buffer sizes, or different quantization formats for the same layer's expert weights.

Common situations: Mixed-precision deployments where one worker was launched with a different quant config; heterogeneous GPUs forcing different dtype strategies; partial rollout of a quantization change during elastic scaling.

Related errors

• Endpoint not ready after {0}s: {1}
• Configuration error: {0}
• IO error: {0}
• chat template error: {0}
• multimodal input is not supported by this chat renderer

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/5f7059da0a6340d6. Report an issue: GitHub.