vllm-project/vllm · error · TemplateError

failed to parse chat_template.json

Error message

failed to parse chat_template.json

What it means

The HF renderer tried to deserialize `chat_template.json` (the multi-template file used by newer transformers models) and serde_json failed; the parse error is the `#[source]`. The file was found and read, but its bytes are not valid JSON.

Source

Thrown at rust/src/chat/src/renderer/hf/error.rs:14

// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: Copyright contributors to the vLLM project

use thiserror::Error as ThisError;

#[derive(Debug, ThisError)]
pub(crate) enum TemplateError {
    #[error("failed to render jinja template")]
    Jinja(#[from] minijinja::Error),
    #[error("failed to read chat template file")]
    ReadTemplateFile(#[source] std::io::Error),
    #[error("chat template looks like a file path but does not exist")]
    MissingTemplatePath,
    #[error("failed to parse chat_template.json")]
    ParseTemplateJson(#[source] serde_json::Error),
    #[error("chat_template.json does not contain a valid template")]
    InvalidTemplateJson,
}

View on GitHub (pinned to c794754062)

Solutions

1. Run `jq . chat_template.json` or `python -m json.tool` on the file to find the exact syntax error location.
2. Re-download the file from the model repo (`huggingface-cli download <repo> chat_template.json --force`).
3. Validate JSON in CI if you edit chat_template.json as part of your deploy pipeline.

Example fix

# before
$ python -m json.tool chat_template.json
# Expecting ',' delimiter (fix the syntax error, or re-download)

# after
$ huggingface-cli download <model-repo> chat_template.json --local-dir . --force
$ python -m json.tool chat_template.json  # OK

let raw = std::fs::read_to_string("chat_template.json")?;
let parsed: serde_json::Value = serde_json::from_str(&raw)?; // fail fast on malformed JSON

match result {
    Err(TemplateError::ParseTemplateJson(err)) => {
        eprintln!("chat_template.json is not valid JSON: {err}; re-download or fix syntax");
        std::process::exit(2);
    }
    other => other?,
}

When it happens

Trigger: Pointing the renderer at a `chat_template.json` that is truncated, has trailing commas/comments, BOM issues, or was corrupted during download; a manually edited file with syntax mistakes.

Common situations: Interrupted `huggingface-cli download` leaving a partial file; hand-editing chat_template.json to add a template; files saved with UTF-8 BOM; symlinks to malformed content.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• chat_template.json does not contain a valid template
• failed to render jinja template
• failed to read chat template file
• chat template looks like a file path but does not exist
• messagepack value decode failed

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/8ed1ec345da4f46f. Report an issue: GitHub.