What causes "request `{request_id}` is already in flight" in vllm-project/vllm?

Use a fresh unique ID per request (uuid::Uuid::new_v4() or a monotonically increasing counter that never repeats)

vllm-project/vllm · error · Error

request `{request_id}` is already in flight

Error message

request `{request_id}` is already in flight

What it means

EngineCoreError::DuplicateRequestId is returned when a generate request is submitted with a `request_id` that is already tracked as in-flight by the client. The client keys active request output streams by request_id, so a duplicate would collide with the existing stream.

Source

Thrown at rust/src/engine-core-client/src/error.rs:69

    UnexpectedHandshakeMessage { message: String },
    #[error("unexpected non-control output on coordinator path: {message}")]
    UnexpectedCoordinatorOutput { message: String },
    #[error("unexpected output on main dispatcher path: {message}")]
    UnexpectedDispatcherOutput { message: String },
    #[error("coordinator requires a Python-compatible two-byte engine id, got {engine_id:?}")]
    UnsupportedCoordinatorEngineId { engine_id: Vec<u8> },
    #[error("unsupported auxiliary frame(s): expected 1 frame, got {frame_count}")]
    UnsupportedAuxFrames { frame_count: usize },
    #[error("external coordinator mode is not implemented yet")]
    UnsupportedExternalCoordinator,
    #[error("unsupported field `{field}` in {context}")]
    UnsupportedField {
        context: &'static str,
        field: &'static str,
    },
    #[error("engine control channel closed unexpectedly: {message}")]
    ControlClosed { message: String },
    #[error("request `{request_id}` is already in flight")]
    DuplicateRequestId { request_id: String },
    #[error(
        "data parallel rank {rank} is not connected to this frontend; connected ranks: {connected_ranks:?}"
    )]
    InvalidDataParallelRank {
        rank: u32,
        connected_ranks: Vec<u32>,
    },
    #[error("engine-core output dispatcher closed: {message}")]
    DispatcherClosed { message: String },
    #[error("engine-core client is closed: {message}")]
    ClientClosed { message: String },
    #[error("request output stream for `{request_id}` closed unexpectedly")]
    RequestStreamClosed { request_id: String },
    #[error("utility call `{method}` failed (call_id={call_id}): {message}")]
    UtilityCallFailed {
        method: String,
        call_id: UtilityCallId,

View on GitHub (pinned to c794754062)

Solutions

Use a fresh unique ID per request (uuid::Uuid::new_v4() or a monotonically increasing counter that never repeats)
Await or abort the in-flight request with the same ID before resubmitting it
If retrying after a failure, ensure the failed request was fully removed from the in-flight map (abort it) before reusing its ID

Example fix

// before
let rid = "req-1".to_string();
for _ in 0..3 { client.generate(rid.clone(), req.clone()).await; }

// after
for _ in 0..3 {
    let rid = uuid::Uuid::new_v4().to_string();
    client.generate(rid, req.clone()).await;
}

Defensive patterns

Strategy: validation

Validate before calling

// Before submitting, ensure the ID is not already in flight
if client.in_flight(&request_id).await {
    return Err(anyhow::anyhow!("request {request_id} still in flight; pick a new id"));
}
client.generate(request_id, req).await

Try / catch

if let Err(vllm_engine_core_client::Error::DuplicateRequestId { request_id }) = res {
    tracing::warn!("{request_id} reused; aborting old and retrying with fresh id");
    client.abort(request_id.clone()).await.ok();
    res = client.generate(new_id(), req).await;
}

Prevention

Always generate request IDs from a UUID v4 generator instead of user-supplied or hardcoded strings
When retrying a failed request, allocate a new request_id rather than reusing the original
Abort the previous request before deliberately reusing its ID

When it happens

Trigger: Calling client.generate() (or the LLM facade) twice with the same explicit request_id before the first request finishes; reusing counters or static IDs across retries instead of fresh unique IDs.

Common situations: Retry logic that resubmits with the same request_id while the original is still streaming; sharing an ID generator that resets; copying example code that hardcodes request_id = "0" or "1" in a loop.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/d985c49f7e26bf9f. Report an issue: GitHub.