vllm-project/vllm · error · vllm_llm::Error

generate request `{request_id}` has an empty prompt_token_id

Error message

generate request `{request_id}` has an empty prompt_token_ids

What it means

llm::Error::EmptyPromptTokenIds is the Rust LLM facade's guard against generate requests that carry an empty `prompt_token_ids` vector. An empty prompt cannot be scheduled by the engine, so the facade rejects it before reaching engine-core.

Source

Thrown at rust/src/llm/src/error.rs:11

// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: Copyright contributors to the vLLM project

use thiserror::Error;

pub type Result<T> = std::result::Result<T, Error>;

/// Public error type for the Rust `llm` facade.
#[derive(Debug, Error)]
pub enum Error {
    #[error("generate request `{request_id}` has an empty prompt_token_ids")]
    EmptyPromptTokenIds { request_id: String },
    #[error("engine-core error")]
    EngineCoreClient(#[from] vllm_engine_core_client::Error),
}

View on GitHub (pinned to c794754062)

Solutions

1. Skip or reject empty inputs before calling generate (validate prompt text/token count)
2. If the prompt is legitimately empty, supply at least one token (e.g. BOS) per the model's expectations
3. Log request_id alongside the check to find which upstream path produces empty token lists

Example fix

// before
let req = GenerateRequest { request_id, prompt_token_ids: tokens, .. };
llm.generate(req).await

// after
if tokens.is_empty() {
    return Err(anyhow!("prompt produced no tokens"));
}
let req = GenerateRequest { request_id, prompt_token_ids: tokens, .. };
llm.generate(req).await

if request.prompt_token_ids.is_empty() {
    return Err(anyhow::anyhow!("refusing to send empty prompt for {}", request.request_id));
}
llm.generate(request).await

fn has_tokens(r: &GenerateRequest) -> bool {
    !r.prompt_token_ids.is_empty()
}

match llm.generate(req).await {
    Err(e @ vllm_llm::Error::EmptyPromptTokenIds { request_id }) => {
        tracing::warn!("skipping empty prompt {request_id}");
        Ok(Default::default())
    }
    other => other,
}

When it happens

Trigger: Building a GenerateRequest with prompt_token_ids: vec![] (e.g. the tokenizer produced no tokens for empty/whitespace input, or tokenization was skipped by mistake) and passing it to the LLM generate API.

Common situations: Passing an empty string or empty messages list that tokenizes to zero tokens; branching that forgets to tokenize; chunked-pipeline bugs producing an empty final chunk.

Related errors

• this model's maximum context length is {max_model_len} token
• duplicate tool name `{name}`
• tool_choice requires at least one available tool
• tool_choice function `{name}` was not found in the available
• failed to build structural tag: {message}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/787a9c05b6916d21. Report an issue: GitHub.