vllm-project/vllm · error · TokenIdsError

token_id(s) {token_ids:?} in {parameter} are out of vocabula

Error message

token_id(s) {token_ids:?} in {parameter} are out of vocabulary. Vocabulary size: {vocab_size}

What it means

One or more token IDs in a token-ID-typed parameter exceed the model's vocabulary size (`sampling_limits.model_vocab_size`). Checked centrally in `validate_param` / `validate_vocab_range` (rust/src/text/src/lower/token_ids.rs and lower.rs) for `allowed_token_ids`, `stop_token_ids`, `logprob_token_ids`, and bad_words-derived IDs; `parameter` names the offending field.

Source

Thrown at rust/src/text/src/lower/token_ids.rs:15

// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: Copyright contributors to the vLLM project

use std::result::Result;

use thiserror::Error;
use vllm_engine_core_client::protocol::sampling::EngineCoreSamplingParams;

use crate::SamplingLimits;

#[derive(Debug, Error)]
pub enum TokenIdsError {
    #[error("allowed_token_ids should not be empty")]
    EmptyAllowedTokenIds,
    #[error(
        "token_id(s) {token_ids:?} in {parameter} are out of vocabulary. \
         Vocabulary size: {vocab_size}"
    )]
    OutOfVocab {
        parameter: &'static str,
        token_ids: Vec<u32>,
        vocab_size: usize,
    },
}

fn validate_param(
    parameter: &'static str,
    token_ids: impl IntoIterator<Item = u32>,
    vocab_size: usize,
) -> Result<(), TokenIdsError> {
    let invalid_token_ids: Vec<_> = token_ids
        .into_iter()
        .filter(|&token_id| token_id as usize >= vocab_size)

View on GitHub (pinned to c794754062)

Solutions

1. Regenerate token IDs with the tokenizer of the currently served model
2. Reference tokens by string (bad_words / stop strings) and let the frontend tokenize
3. Filter cached IDs against the server's reported vocab size before sending

Example fix

// before
req.stop_token_ids = Some(vec![151645, 999999]); // 999999 > vocab

// after
req.stop_token_ids = Some(vec![151645]);
// or express by text
req.stop = Some(vec!["<|im_end|>".into()]);

let vocab = server_vocab_size; // from model metadata
for (name, ids) in [("stop_token_ids", &stop_token_ids), ("allowed_token_ids", &allowed_token_ids), ("logprob_token_ids", &logprob_token_ids)] {
    assert!(ids.iter().all(|&t| (t as usize) < vocab),
        "{name} contains IDs >= vocab size {vocab}");
}

fn is_out_of_vocab(e: &Error) -> bool {
    matches!(e, Error::TokenIds(TokenIdsError::OutOfVocab { .. }))
}

match err {
    Error::TokenIds(TokenIdsError::OutOfVocab { parameter, token_ids, vocab_size }) =>
        bad_request(format!("{parameter} {token_ids:?} >= vocab {vocab_size}; regenerate IDs with the served tokenizer")),
    _ => /* ... */
}

When it happens

Trigger: Passing token IDs obtained from a different tokenizer/model (e.g. IDs >= vocab_size of the served model) in stop_token_ids, allowed_token_ids, or logprob_token_ids; bad_words strings tokenized to IDs not in the served vocab.

Common situations: Swapping served models while caching token IDs client-side; hardcoding token IDs (e.g. an EOS id like 151645) valid only for one tokenizer family; mixing Qwen/Llama vocabularies.

Related errors

• allowed_token_ids should not be empty
• this model's maximum context length is {max_model_len} token
• max_logprobs must be non-negative or -1, got {}
• data parallel size must be at least 1
• engine start index does not fit usize

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/096cd879fa194c7d. Report an issue: GitHub.