What causes "tool parser parsing failed: {message}" in vllm-project/vllm?

Inspect `message` for the exact parse failure position/cause

vllm-project/vllm · warning · ToolParserError

tool parser parsing failed: {message}

Error message

tool parser parsing failed: {message}

What it means

ToolParserError::ParsingFailed is the generic failure a tool parser raises when it cannot make progress on the streamed tool-call syntax — malformed JSON arguments, unbalanced function-call delimiters, or state-machine violations while consuming deltas. The `message` string carries the parser-specific detail.

Source

Thrown at rust/src/parser/src/tool/error.rs:14

// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: Copyright contributors to the vLLM project

use thiserror::Error;
use thiserror_ext::Macro;

/// Result alias for tool parser operations.
pub type Result<T> = std::result::Result<T, ToolParserError>;

/// Errors produced while creating or running tool parsers.
#[derive(Debug, Error, Macro)]
#[thiserror_ext(macro(path = "crate::tool::error"))]
pub enum ToolParserError {
    #[error("tool parser parsing failed: {message}")]
    ParsingFailed { message: String },
    #[error(
        "`{name}` only provides a unified parser; the same reasoning parser and tool parser should be specified together"
    )]
    DummyUnifiedParser { name: String },
}

View on GitHub (pinned to c794754062)

Solutions

Inspect `message` for the exact parse failure position/cause
Lower temperature / tighten sampling so the model emits well-formed tool calls
Ensure the tool parser matches the model's tool-call format (e.g. hermes parser with hermes-style templates)
Handle the error per-request and surface it to the client rather than aborting the whole stream

Defensive patterns

Strategy: try-catch

Type guard

fn is_tool_parse_failed(e: &ToolParserError) -> bool {
    matches!(e, ToolParserError::ParsingFailed { .. })
}

Try / catch

match tool_parser.push(delta) {
    Err(ToolParserError::ParsingFailed { message }) => {
        tracing::warn!("tool call malformed: {message}");
        stream.emit_tool_error(request_id, message); // degrade this call, keep stream alive
    }
    r => r?,
}

Prevention

Surface tool-parse failures per-request instead of aborting the connection
Keep sampling temperature low for tool-calling workloads
Match the tool parser to the model's tool-call template exactly

When it happens

Trigger: Model output that drifts from the tool-call grammar the parser expects (broken JSON in arguments, missing closing tags like </tool_call>, interleaved reasoning text inside tool call blocks); streaming deltas arriving out of the assumed order.

Common situations: Small or lightly fine-tuned models emitting malformed tool calls; prompt templates that corrupt the tool-call format; aggressive sampling parameters (high temperature) producing invalid syntax; switching tool parser without switching chat template.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/e954b609d61a85f4. Report an issue: GitHub.