{"record":{"id":"18dfc97cbb3db09e","repo":"vllm-project/vllm","slug":"chat-request-stream-request-id-closed-before-t","errorCode":null,"errorMessage":"chat request stream `{request_id}` closed before terminal output","messagePattern":"chat request stream `(.+?)` closed before terminal output","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"rust/src/chat/src/error.rs","lineNumber":70,"sourceCode":"    },\n    #[error(\n        \"gpt_oss uses native Harmony output parsing; generic {kind} parser override `{selection}` is not supported\"\n    )]\n    HarmonyParserOverrideUnsupported {\n        kind: &'static str,\n        selection: String,\n    },\n    #[error(\"harmony output parsing failed\")]\n    HarmonyOutputParsing {\n        #[source]\n        error: BoxedError,\n    },\n    #[error(\n        \"this model's maximum context length is {max_model_len} tokens, \\\n         but the prompt contains {prompt_len} input tokens\"\n    )]\n    PromptTooLong { max_model_len: u32, prompt_len: u32 },\n    #[error(\"chat request stream `{request_id}` closed before terminal output\")]\n    StreamClosedBeforeTerminalOutput { request_id: String },\n    #[error(\"tool call stream state is inconsistent: {message}\")]\n    ToolCallStreamInvariant { message: String },\n    #[error(\"duplicate tool name `{name}`\")]\n    DuplicateToolName { name: String },\n    #[error(\"tool_choice requires at least one available tool\")]\n    ToolChoiceRequiresTools,\n    #[error(\"tool_choice function `{name}` was not found in the available tools\")]\n    ToolChoiceFunctionNotFound { name: String },\n    #[error(\"failed to build structural tag: {message}\")]\n    StructuralTag { message: String },\n    #[error(transparent)]\n    Text(#[from] vllm_text::Error),\n    #[error(transparent)]\n    Tokenizer(#[from] vllm_tokenizer::TokenizerError),\n}\n\npub type Result<T> = std::result::Result<T, Error>;","sourceCodeStart":52,"sourceCodeEnd":88,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/rust/src/chat/src/error.rs#L52-L88","documentation":"The chat request's response stream ended (client disconnect or stream terminator) before any terminal output event (e.g. finish/stop) was produced — see `rust/src/chat/src/stream.rs:114` and `:158`. The `request_id` field identifies which request died. It signals a protocol/lifecycle violation: a stream must terminate with a terminal message.","triggerScenarios":"SSE/streaming chat response where the underlying transport closes or the generator is dropped before emitting the terminal chunk; client aborts the HTTP request mid-stream; internal task handling the stream panics or is cancelled.","commonSituations":"Client timeouts (proxy idle timeouts killing SSE connections); load balancer cutting long streams; bugs where the stream handler returns early without a finish event; disconnect-on-first-token patterns.","solutions":["Match the `request_id` to server logs to find which request/stream aborted.","Raise client and intermediary (proxy/LB) read timeouts for streaming endpoints.","If your client intentionally aborts streams, treat this error as expected and ignore/log it rather than failing.","Check for panics or early returns in custom stream middleware that skip the terminal event."],"exampleFix":"// before\nlet stream = client.chat_stream(req); // client drops stream early\n\n// after\nlet mut stream = client.chat_stream(req)?;\nwhile let Some(ev) = stream.next().await {\n    handle(ev)?; // consume until the terminal event before dropping\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"if matches!(err, vllm_chat::Error::StreamClosedBeforeTerminalOutput { .. }) {\n    // client disconnect or upstream abort: log at info, do not alert\n    tracing::info!(?err, \"stream aborted before terminal output\");\n    return Ok(());\n}","preventionTips":["Consume streams to the terminal event before dropping them in client code.","Set proxy/LB idle timeouts above your longest expected inter-chunk gap for SSE.","Correlate request_id from the error with server logs instead of guessing."],"tags":["rust","streaming","sse","lifecycle","chat"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}