vllm-project/vllm · error · LogprobsError

when both logprobs and logprob_token_ids are set, logprobs m

Error message

when both logprobs and logprob_token_ids are set, logprobs must equal len(logprob_token_ids). Got logprobs={logprobs}, len(logprob_token_ids)={num_token_ids}.

What it means

When both `logprobs` and `logprob_token_ids` are present in a request, the Rust frontend requires `logprobs == len(logprob_token_ids)` (rust/src/text/src/lower/logprobs.rs:94-101), matching the Python API contract that the count parameter describes exactly how many per-position logprobs are returned.

Source

Thrown at rust/src/text/src/lower/logprobs.rs:33

    #[error("{parameter} must be non-negative or -1, got {value}")]
    InvalidCount { parameter: &'static str, value: i32 },
    #[error(
        "requested {parameter} of {requested}, which is greater than max allowed: {max_allowed}"
    )]
    TooManyCount {
        parameter: &'static str,
        requested: usize,
        max_allowed: usize,
    },
    #[error(
        "requested logprob_token_ids of length {requested}, \
         which is greater than max allowed: {max_allowed}"
    )]
    TooManyTokenIds {
        requested: usize,
        max_allowed: usize,
    },
    #[error(
        "when both logprobs and logprob_token_ids are set, logprobs must equal \
         len(logprob_token_ids). Got logprobs={logprobs}, len(logprob_token_ids)={num_token_ids}."
    )]
    TokenIdsMismatch { logprobs: i32, num_token_ids: usize },
}

/// Validate logprobs count sampling parameters.
pub(super) fn validate_logprobs(
    logprobs: Option<i32>,
    prompt_logprobs: Option<i32>,
    logprob_token_ids: Option<&[u32]>,
    sampling_limits: SamplingLimits,
) -> Result<(), LogprobsError> {
    let vocab_size = sampling_limits.model_vocab_size;
    let max_logprobs =
        normalize_logprobs_count(sampling_limits.max_logprobs, vocab_size, "max_logprobs")?;

    validate_logprobs_count(logprobs, max_logprobs, vocab_size, "logprobs")?;

View on GitHub (pinned to c794754062)

Solutions

Set logprobs to exactly len(logprob_token_ids)
Or drop one of the two fields: omitting logprobs skips the check

Example fix

# before
logprobs = 10
logprob_token_ids = [1, 2, 3]

# after
logprobs = 3
logprob_token_ids = [1, 2, 3]

Defensive patterns

Strategy: validation

Validate before calling

if let (Some(lp), Some(ids)) = (request.logprobs, request.logprob_token_ids.as_deref()) {
    assert!(lp == ids.len() as i32,
        "logprobs ({lp}) must equal len(logprob_token_ids) ({})", ids.len());
}

Type guard

fn is_token_ids_mismatch(e: &Error) -> bool {
    matches!(e, Error::Logprobs(LogprobsError::TokenIdsMismatch { .. }))
}

Try / catch

match err {
    Error::Logprobs(LogprobsError::TokenIdsMismatch { logprobs, num_token_ids }) =>
        bad_request(format!("set logprobs={num_token_ids} or drop it (got {logprobs})")),
    _ => /* ... */
}

Prevention

Derive logprobs from the list length in one place: request.logprobs = Some(ids.len() as i32)
Never hand-edit the two fields independently

When it happens

Trigger: Sending e.g. `logprobs: 5` together with a `logprob_token_ids` array of length 3 (any mismatch triggers it).

Common situations: Independently tuning two fields that must agree; patching requests in middleware that updates one field but not the other; migrating from an API that allowed them to differ.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/6e53343fe4a60b1f. Report an issue: GitHub.