vllm-project/vllm · error · Error

text request stream `{request_id}` closed before terminal ou

Error message

text request stream `{request_id}` closed before terminal output

What it means

The decoded output stream for a request ended without ever emitting a terminal event (finish reason / usage). Raised at the end of the decode loop in rust/src/text/src/output/decoded.rs:301 and as a defensive branch in output/mod.rs:104 (there noted as effectively unreachable because the underlying engine stream reports its own error on unexpected close). It means the request produced partial or no output and then the stream closed cleanly without a finish marker.

Source

Thrown at rust/src/text/src/error.rs:40

         but the prompt contains {prompt_len} input tokens"
    )]
    PromptTooLong { max_model_len: u32, prompt_len: u32 },
    #[error(transparent)]
    Logprobs(#[from] LogprobsError),
    #[error(transparent)]
    TokenIds(#[from] TokenIdsError),
    #[error(transparent)]
    SamplingParams(#[from] SamplingParamsError),
    #[error(
        "`min_tokens` must be less than or equal to `max_tokens`, \
         got min_tokens={min_tokens}, max_tokens={max_tokens}"
    )]
    MinTokensExceedsMaxTokens { min_tokens: u32, max_tokens: u32 },
    #[error("`thinking_token_budget` must be a non-negative integer or -1 for unlimited.")]
    InvalidThinkingTokenBudget,
    #[error("invalid repetition detection params: {message}")]
    InvalidRepetitionDetection { message: String },
    #[error("text request stream `{request_id}` closed before terminal output")]
    StreamClosedBeforeTerminalOutput { request_id: String },
    #[error(transparent)]
    Llm(#[from] LlmError),
    #[error(transparent)]
    EngineCore(#[from] EngineCoreError),
}

pub type Result<T> = std::result::Result<T, Error>;

impl Error {
    /// Whether this error represents invalid user request parameters.
    pub fn is_request_validation_error(&self) -> bool {
        match self {
            Self::PromptTooLong { .. }
            | Self::EmptyPromptTokenIds { .. }
            | Self::EmptyStopString { .. }
            | Self::Logprobs(_)
            | Self::TokenIds(_)

View on GitHub (pinned to c794754062)

Solutions

1. Retry the request; this reflects an abnormal termination, not bad parameters
2. Check engine-core logs for the same request_id to find the root cause (abort, crash, or dropped terminal event)
3. If reproducible, capture the request_id and report it — a clean stream close without terminal output indicates a frontend/engine contract bug

fn is_stream_closed(e: &Error) -> bool {
    matches!(e, Error::StreamClosedBeforeTerminalOutput { .. })
}

match collect_output(stream).await {
    Err(Error::StreamClosedBeforeTerminalOutput { request_id }) => {
        tracing::warn!(%request_id, "stream closed before terminal output; retrying");
        retry_with_backoff().await
    }
    other => other,
}

When it happens

Trigger: Engine-core ends the output stream for a request without a finish event: engine-side cancellation/abort that closes the stream quietly, a detokenizer bug that never forwards the terminal event, or a client dropping the request mid-stream causing the collector to observe end-of-stream first.

Common situations: Long-running generations interrupted by server shutdown or preemption; bugs during upgrades of the Rust frontend where a new event type is not recognized as terminal; stress tests that abort requests concurrently.

Related errors

• request output stream for `{request_id}` closed unexpectedly
• harmony output parsing failed
• chat request stream `{request_id}` closed before terminal ou
• tool call stream state is inconsistent: {message}
• engine control channel closed unexpectedly: {message}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/a5e1183711778f6c. Report an issue: GitHub.