{"record":{"id":"1682fa493f5807c3","repo":"vllm-project/vllm","slug":"tool-call-stream-state-is-inconsistent-message","errorCode":null,"errorMessage":"tool call stream state is inconsistent: {message}","messagePattern":"tool call stream state is inconsistent: (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"rust/src/chat/src/error.rs","lineNumber":72,"sourceCode":"        \"gpt_oss uses native Harmony output parsing; generic {kind} parser override `{selection}` is not supported\"\n    )]\n    HarmonyParserOverrideUnsupported {\n        kind: &'static str,\n        selection: String,\n    },\n    #[error(\"harmony output parsing failed\")]\n    HarmonyOutputParsing {\n        #[source]\n        error: BoxedError,\n    },\n    #[error(\n        \"this model's maximum context length is {max_model_len} tokens, \\\n         but the prompt contains {prompt_len} input tokens\"\n    )]\n    PromptTooLong { max_model_len: u32, prompt_len: u32 },\n    #[error(\"chat request stream `{request_id}` closed before terminal output\")]\n    StreamClosedBeforeTerminalOutput { request_id: String },\n    #[error(\"tool call stream state is inconsistent: {message}\")]\n    ToolCallStreamInvariant { message: String },\n    #[error(\"duplicate tool name `{name}`\")]\n    DuplicateToolName { name: String },\n    #[error(\"tool_choice requires at least one available tool\")]\n    ToolChoiceRequiresTools,\n    #[error(\"tool_choice function `{name}` was not found in the available tools\")]\n    ToolChoiceFunctionNotFound { name: String },\n    #[error(\"failed to build structural tag: {message}\")]\n    StructuralTag { message: String },\n    #[error(transparent)]\n    Text(#[from] vllm_text::Error),\n    #[error(transparent)]\n    Tokenizer(#[from] vllm_tokenizer::TokenizerError),\n}\n\npub type Result<T> = std::result::Result<T, Error>;\n\nimpl Error {","sourceCodeStart":54,"sourceCodeEnd":90,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/rust/src/chat/src/error.rs#L54-L90","documentation":"Internal invariant violation in tool-call stream assembly: while folding deltas into tool calls, the state machine reached a combination it cannot represent (raised at `output/structured.rs:134` and `output/default/unified.rs:177/185`). This is a defect-level error — either a parser bug or non-conforming model output — not a user input error (not listed in `is_request_validation_error`).","triggerScenarios":"Streaming with a tool-call parser where deltas arrive out of the expected order (e.g. argument delta for a tool index that was never opened, or two `name` deltas for one index); model emitting malformed partial tool-call JSON that the incremental decoder cannot merge.","commonSituations":"Custom or newly added tool parser with a buggy incremental mode; a model fine-tune that emits interleaved/duplicated tool-call tags; switching parsers (e.g. from hermes to mistral style) against a model trained for the other format.","solutions":["Capture the full raw delta sequence for the failing request to see which invariant (index/name/arguments ordering) broke.","Try a different tool-call parser that matches the model's trained format.","Retry the request non-streaming — the full-text parser path avoids incremental state entirely.","If it reproduces with stock models/parsers, file a bug with the `message` field content and delta trace (it names the exact invariant)."],"exampleFix":"// before\nparser: \"hermes\" // model emits mistral-style [TOOL_CALLS] tags\n\n// after\nparser: \"mistral\" // parser format matches model training","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"match stream_result {\n    Err(e @ vllm_chat::Error::ToolCallStreamInvariant { .. }) => {\n        tracing::warn!(%e, \"incremental tool-call decode failed; retrying non-streaming\");\n        client.chat(request.with_stream(false)).await // full-text parse path\n    }\n    other => other,\n}","preventionTips":["Choose the tool-call parser that matches the model's trained tag format.","Keep a non-streaming fallback path for tool-heavy traffic.","Report reproducible invariant messages upstream — they indicate parser bugs."],"tags":["rust","tool-calls","streaming","invariant","internal"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}