vllm-project/vllm · error · Error

request output stream for `{request_id}` closed unexpectedly

Error message

request output stream for `{request_id}` closed unexpectedly

What it means

EngineCoreError::RequestStreamClosed indicates the per-request output stream for `request_id` ended before the request reached a terminal state (finish/final output). This happens when the engine-core side drops that request's output channel without sending the completion event.

Source

Thrown at rust/src/engine-core-client/src/error.rs:82

        context: &'static str,
        field: &'static str,
    },
    #[error("engine control channel closed unexpectedly: {message}")]
    ControlClosed { message: String },
    #[error("request `{request_id}` is already in flight")]
    DuplicateRequestId { request_id: String },
    #[error(
        "data parallel rank {rank} is not connected to this frontend; connected ranks: {connected_ranks:?}"
    )]
    InvalidDataParallelRank {
        rank: u32,
        connected_ranks: Vec<u32>,
    },
    #[error("engine-core output dispatcher closed: {message}")]
    DispatcherClosed { message: String },
    #[error("engine-core client is closed: {message}")]
    ClientClosed { message: String },
    #[error("request output stream for `{request_id}` closed unexpectedly")]
    RequestStreamClosed { request_id: String },
    #[error("utility call `{method}` failed (call_id={call_id}): {message}")]
    UtilityCallFailed {
        method: String,
        call_id: UtilityCallId,
        message: String,
    },
    #[error("utility call `{method}` returned an invalid result (call_id={call_id}): {message}")]
    UtilityResultDecode {
        method: String,
        call_id: UtilityCallId,
        message: String,
    },
    #[error("utility call `{method}` closed unexpectedly (call_id={call_id})")]
    UtilityCallClosed { method: String, call_id: u64 },
    #[error("utility call `{method}` returned inconsistent results across engines: {values}")]
    InconsistentUtilityResults { method: String, values: String },

View on GitHub (pinned to c794754062)

Solutions

Check engine-core logs for what happened to this request_id (abort, preemption failure, crash)
Retry the request with the same inputs using a NEW request_id (the old ID may still be registered as in-flight)
If it reproduces deterministically, capture the request and report it as an engine-core streaming bug

Defensive patterns

Strategy: retry

Type guard

pub fn is_request_stream_closed(e: &vllm_engine_core_client::Error) -> bool {
    matches!(e, vllm_engine_core_client::Error::RequestStreamClosed { .. })
}

Try / catch

if matches!(err, vllm_engine_core_client::Error::RequestStreamClosed { .. }) && attempt < MAX {
    let rid = uuid::Uuid::new_v4().to_string(); // new id: old one may still be registered
    return retry_generate(rid, req, attempt + 1).await;
}

Prevention

Retry with a fresh request_id since the original may remain marked in-flight
Set client-side timeouts on streams so silent closures are detected promptly
Track finish reasons; treat stream end without a terminal event as an error, not EOF

When it happens

Trigger: Streaming a generate() result and the engine-core aborts, evicts, or loses the request (e.g. engine crash, forced flush, scheduler drop) so the stream ends without a FinishReason-bearing final message.

Common situations: Engine-core restart or crash mid-generation; request aborted by server-side policy while the consumer still polls the stream; bugs where the final output frame is not enqueued.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/86bbd829585a96d8. Report an issue: GitHub.