vllm-project/vllm · error · Error

chat template error: {0}

Error message

chat template error: {0}

What it means

For stateless elastic EP (StatelessGroupCoordinator), only the torch_nccl, pynccl, and nixl backends are implemented; torch_gloo and others rely on process-group state that stateless ranks do not share. The requested backend string is not in that set.

Source

Thrown at rust/src/chat/src/error.rs:18

// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: Copyright contributors to the vLLM project

use thiserror::Error;
use thiserror_ext::{AsReport as _, Macro};

type BoxedError = Box<dyn std::error::Error + Send + Sync>;

#[derive(Debug, Error, Macro)]
#[thiserror_ext(macro(path = "crate::error"))]
pub enum Error {
    #[error("chat request must contain at least one message")]
    EmptyMessages,
    #[error("cannot continue the final message when the last message is not from the assistant")]
    ContinueFinalAssistantWithoutFinalAssistant,
    #[error("chat template is required but none was configured")]
    MissingChatTemplate,
    #[error("chat template error: {0}")]
    ChatTemplate(String),
    #[error("multimodal input is not supported by this chat renderer")]
    UnsupportedMultimodalRenderer,
    #[error("unsupported multimodal content: {0}")]
    UnsupportedMultimodalContent(&'static str),
    #[error("`{modality}` input is not supported by this model")]
    UnsupportedModality { modality: String },
    #[error("At most {limit} {modality}(s) may be provided in one prompt.")]
    MmLimitExceeded { modality: String, limit: usize },
    #[error("multimodal preprocessing error: {0}")]
    Multimodal(#[message] String),
    #[error("{kind} parsing is not available for model `{model_id}`")]
    ParserUnavailableForModel {
        kind: &'static str,
        model_id: String,
    },
    #[error("{kind} parsing is disabled by frontend configuration")]
    ParserDisabled { kind: &'static str },

View on GitHub (pinned to c794754062)

Solutions

1. Use 'torch_nccl', 'pynccl', or 'nixl' for stateless elastic EP (torch_nccl is transparently forced to pynccl anyway)
2. If you truly need gloo, run the non-stateless EPLB path instead of StatelessGroupCoordinator

Example fix

# before
backend = "torch_gloo"  # stateless coordinator

# after
backend = "pynccl"

from vllm.distributed.parallel_state import StatelessGroupCoordinator
if isinstance(group_coordinator, StatelessGroupCoordinator):
    assert backend in ('torch_nccl', 'pynccl', 'nixl'), 'stateless EP supports only these backends'

def stateless_safe_backend(backend: str) -> bool:
    return backend in ('torch_nccl', 'pynccl', 'nixl')

When it happens

Trigger: Passing backend='torch_gloo' (or any other string) to the EPLB communicator factory while running under a StatelessGroupCoordinator, i.e. stateless/multi-process elastic EP mode.

Common situations: Copying a stateful EPLB configuration (which allows torch_gloo) into a stateless elastic EP deployment; custom orchestration code hardcoding a backend name.

Related errors

• Endpoint not ready after {0}s: {1}
• Backend error: {0}
• multimodal input is not supported by this chat renderer
• At most {limit} {modality}(s) may be provided in one prompt.
• Configuration error: {0}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/8a0312497381d811. Report an issue: GitHub.