vllm-project/vllm · error · LogprobsError

requested logprob_token_ids of length {requested}, which is

Error message

requested logprob_token_ids of length {requested}, which is greater than max allowed: {max_allowed}

What it means

`logprob_token_ids` (the list of specific token IDs to return logprobs for) exceeds the hard frontend cap `SamplingLimits::MAX_LOGPROB_TOKEN_IDS` (rust/src/text/src/lower/logprobs.rs:87-92). Unlike counts, this cap is a compile-time constant of the Rust text frontend, not a server knob.

Source

Thrown at rust/src/text/src/lower/logprobs.rs:25

//! passed through to engine-core.

use thiserror::Error;

use crate::backend::SamplingLimits;

#[derive(Debug, Error)]
pub enum LogprobsError {
    #[error("{parameter} must be non-negative or -1, got {value}")]
    InvalidCount { parameter: &'static str, value: i32 },
    #[error(
        "requested {parameter} of {requested}, which is greater than max allowed: {max_allowed}"
    )]
    TooManyCount {
        parameter: &'static str,
        requested: usize,
        max_allowed: usize,
    },
    #[error(
        "requested logprob_token_ids of length {requested}, \
         which is greater than max allowed: {max_allowed}"
    )]
    TooManyTokenIds {
        requested: usize,
        max_allowed: usize,
    },
    #[error(
        "when both logprobs and logprob_token_ids are set, logprobs must equal \
         len(logprob_token_ids). Got logprobs={logprobs}, len(logprob_token_ids)={num_token_ids}."
    )]
    TokenIdsMismatch { logprobs: i32, num_token_ids: usize },
}

/// Validate logprobs count sampling parameters.
pub(super) fn validate_logprobs(
    logprobs: Option<i32>,
    prompt_logprobs: Option<i32>,

View on GitHub (pinned to c794754062)

Solutions

1. Trim logprob_token_ids to the cap; request only the token IDs you actually inspect
2. If you genuinely need broad coverage, use logprobs (top-K count) instead of an explicit ID list

Example fix

// before
request.logprob_token_ids = all_vocab_ids; // huge list

// after
request.logprob_token_ids = watchlist.slice(0, MAX_LOGPROB_TOKEN_IDS);

const MAX_LOGPROB_TOKEN_IDS: usize = SamplingLimits::MAX_LOGPROB_TOKEN_IDS;
assert!(logprob_token_ids.len() <= MAX_LOGPROB_TOKEN_IDS,
    "too many logprob_token_ids: {}", logprob_token_ids.len());

fn is_too_many_token_ids(e: &Error) -> bool {
    matches!(e, Error::Logprobs(LogprobsError::TooManyTokenIds { .. }))
}

match err {
    Error::Logprobs(LogprobsError::TooManyTokenIds { requested, max_allowed }) =>
        bad_request(format!("logprob_token_ids len {requested} > {max_allowed}")),
    _ => /* ... */
}

When it happens

Trigger: Passing a `logprob_token_ids` array longer than MAX_LOGPROB_TOKEN_IDS in a request.

Common situations: Building large curated token watch-lists (e.g. all tokens of several words); attempting to use logprob_token_ids as a full-vocabulary workaround when logprobs is capped.

Related errors

• max_logprobs must be non-negative or -1, got {}
• max_logprobs must be non-negative or -1
• {parameter} must be non-negative or -1, got {value}
• requested {parameter} of {requested}, which is greater than
• when both logprobs and logprob_token_ids are set, logprobs m

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/01ea20cd62317417. Report an issue: GitHub.