vllm-project/vllm · error · TemplateError

chat_template.json does not contain a valid template

Error message

chat_template.json does not contain a valid template

What it means

`chat_template.json` parsed as JSON but did not contain a valid template where one is required (`template.rs:72` fails `serde_json::from_value` into the expected shape and maps the failure to this variant). The file's structure is wrong — e.g. missing the expected template field or holding an unexpected type.

Source

Thrown at rust/src/chat/src/renderer/hf/error.rs:16

// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: Copyright contributors to the vLLM project

use thiserror::Error as ThisError;

#[derive(Debug, ThisError)]
pub(crate) enum TemplateError {
    #[error("failed to render jinja template")]
    Jinja(#[from] minijinja::Error),
    #[error("failed to read chat template file")]
    ReadTemplateFile(#[source] std::io::Error),
    #[error("chat template looks like a file path but does not exist")]
    MissingTemplatePath,
    #[error("failed to parse chat_template.json")]
    ParseTemplateJson(#[source] serde_json::Error),
    #[error("chat_template.json does not contain a valid template")]
    InvalidTemplateJson,
}

View on GitHub (pinned to c794754062)

Solutions

1. Inspect chat_template.json and confirm it contains the template kind being requested (base chat template and any named variants).
2. Fall back to an inline template override or a template file instead of chat_template.json.
3. If a specific named variant is missing, use the model's default template rather than requesting the variant.

Example fix

// before
{"tool_use_template": "..."} // renderer needs base "chat_template" entry

// after
{"chat_template": "...", "tool_use_template": "..."}

let json: serde_json::Value = serde_json::from_str(&raw)?;
if json.get("chat_template").or_else(|| json.get("default")).is_none() {
    return Err("chat_template.json has no usable template entry");
}

match result {
    Err(TemplateError::InvalidTemplateJson) => {
        eprintln!("chat_template.json lacks the required template entry; override with an explicit template");
        std::process::exit(2);
    }
    other => other?,
}

When it happens

Trigger: A chat_template.json that is valid JSON but, at the point a specific template is requested, the corresponding entry is absent or not a string/map the renderer can use — e.g. requesting the tool template when the JSON only defines a base chat_template.

Common situations: Model repos whose chat_template.json layout differs from the transformers version the renderer targets; hand-rolled chat_template.json with only custom templates; requesting a named template variant that the model author never shipped.

Related errors

• failed to parse chat_template.json
• this model's maximum context length is {max_model_len} token
• duplicate tool name `{name}`
• tool_choice requires at least one available tool
• tool_choice function `{name}` was not found in the available

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/5e50437031b775a1. Report an issue: GitHub.