{"record":{"id":"fbfc97fb0b762734","repo":"vllm-project/vllm","slug":"harmony-output-parsing-failed","errorCode":null,"errorMessage":"harmony output parsing failed","messagePattern":"harmony output parsing failed","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"rust/src/chat/src/error.rs","lineNumber":60,"sourceCode":"        kind: &'static str,\n        name: String,\n        available_names: Vec<String>,\n    },\n    #[error(\"failed to initialize {kind} parser `{name}`\")]\n    ParserInitialization {\n        kind: &'static str,\n        name: String,\n        #[source]\n        error: BoxedError,\n    },\n    #[error(\n        \"gpt_oss uses native Harmony output parsing; generic {kind} parser override `{selection}` is not supported\"\n    )]\n    HarmonyParserOverrideUnsupported {\n        kind: &'static str,\n        selection: String,\n    },\n    #[error(\"harmony output parsing failed\")]\n    HarmonyOutputParsing {\n        #[source]\n        error: BoxedError,\n    },\n    #[error(\n        \"this model's maximum context length is {max_model_len} tokens, \\\n         but the prompt contains {prompt_len} input tokens\"\n    )]\n    PromptTooLong { max_model_len: u32, prompt_len: u32 },\n    #[error(\"chat request stream `{request_id}` closed before terminal output\")]\n    StreamClosedBeforeTerminalOutput { request_id: String },\n    #[error(\"tool call stream state is inconsistent: {message}\")]\n    ToolCallStreamInvariant { message: String },\n    #[error(\"duplicate tool name `{name}`\")]\n    DuplicateToolName { name: String },\n    #[error(\"tool_choice requires at least one available tool\")]\n    ToolChoiceRequiresTools,\n    #[error(\"tool_choice function `{name}` was not found in the available tools\")]","sourceCodeStart":42,"sourceCodeEnd":78,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/rust/src/chat/src/error.rs#L42-L78","documentation":"Top-level wrapper for failures while decoding gpt-oss model output with the native Harmony parser (the harmony message format with analysis/channel tags). The concrete failure lives in the chained `#[source] BoxedError` (the harmony crate's own error). It means the token stream did not conform to the Harmony grammar the parser expects.","triggerScenarios":"Streaming or non-streaming completion from a gpt-oss model where the generated text cannot be parsed into Harmony events — malformed channel tags, truncated harmony payloads, or a reasoning/assistant segment that fails harmony decoding.","commonSituations":"Output truncated by max_tokens mid-harmony-payload; sampling parameters (very high temperature, bad grammar constraints) producing degenerate output; a fine-tune of gpt-oss that drifted from the harmony format.","solutions":["Read the chained source error to identify which harmony stage failed.","Increase `max_tokens` / max_model_len so harmony payloads are not truncated mid-stream.","Reset sampling parameters to the gpt-oss recommended defaults and re-test.","If persistent on every request, verify the model weights/config actually correspond to a harmony-format gpt-oss checkpoint."],"exampleFix":"// before\nSamplingParams { temperature: 1.5, max_tokens: Some(16), .. } // truncates harmony output\n\n// after\nSamplingParams { temperature: 1.0, max_tokens: Some(1024), .. }","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"match result {\n    Err(vllm_chat::Error::HarmonyOutputParsing { error }) => {\n        tracing::warn!(?error, \"harmony parse failed; retrying once with lower temperature\");\n        retry_with_conservative_sampling().await\n    }\n    other => other,\n}","preventionTips":["Use gpt-oss recommended sampling settings (temperature 1.0, top_p 1.0).","Set max_tokens generously so Harmony payloads are not truncated mid-stream.","Log the raw output alongside the parse failure for diagnosis."],"tags":["rust","gpt-oss","harmony","parsing","streaming"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}