{"record":{"id":"096cd879fa194c7d","repo":"vllm-project/vllm","slug":"token-id-s-token-ids-in-parameter-are-out-o","errorCode":null,"errorMessage":"token_id(s) {token_ids:?} in {parameter} are out of vocabulary. Vocabulary size: {vocab_size}","messagePattern":"token_id\\(s\\) (.+?) in (.+?) are out of vocabulary\\. Vocabulary size: (.+?)","errorType":"validation","errorClass":"TokenIdsError","httpStatus":400,"severity":"error","filePath":"rust/src/text/src/lower/token_ids.rs","lineNumber":15,"sourceCode":"// SPDX-License-Identifier: Apache-2.0\n// SPDX-FileCopyrightText: Copyright contributors to the vLLM project\n\nuse std::result::Result;\n\nuse thiserror::Error;\nuse vllm_engine_core_client::protocol::sampling::EngineCoreSamplingParams;\n\nuse crate::SamplingLimits;\n\n#[derive(Debug, Error)]\npub enum TokenIdsError {\n    #[error(\"allowed_token_ids should not be empty\")]\n    EmptyAllowedTokenIds,\n    #[error(\n        \"token_id(s) {token_ids:?} in {parameter} are out of vocabulary. \\\n         Vocabulary size: {vocab_size}\"\n    )]\n    OutOfVocab {\n        parameter: &'static str,\n        token_ids: Vec<u32>,\n        vocab_size: usize,\n    },\n}\n\nfn validate_param(\n    parameter: &'static str,\n    token_ids: impl IntoIterator<Item = u32>,\n    vocab_size: usize,\n) -> Result<(), TokenIdsError> {\n    let invalid_token_ids: Vec<_> = token_ids\n        .into_iter()\n        .filter(|&token_id| token_id as usize >= vocab_size)","sourceCodeStart":1,"sourceCodeEnd":33,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/rust/src/text/src/lower/token_ids.rs#L1-L33","documentation":"One or more token IDs in a token-ID-typed parameter exceed the model's vocabulary size (`sampling_limits.model_vocab_size`). Checked centrally in `validate_param` / `validate_vocab_range` (rust/src/text/src/lower/token_ids.rs and lower.rs) for `allowed_token_ids`, `stop_token_ids`, `logprob_token_ids`, and bad_words-derived IDs; `parameter` names the offending field.","triggerScenarios":"Passing token IDs obtained from a different tokenizer/model (e.g. IDs >= vocab_size of the served model) in stop_token_ids, allowed_token_ids, or logprob_token_ids; bad_words strings tokenized to IDs not in the served vocab.","commonSituations":"Swapping served models while caching token IDs client-side; hardcoding token IDs (e.g. an EOS id like 151645) valid only for one tokenizer family; mixing Qwen/Llama vocabularies.","solutions":["Regenerate token IDs with the tokenizer of the currently served model","Reference tokens by string (bad_words / stop strings) and let the frontend tokenize","Filter cached IDs against the server's reported vocab size before sending"],"exampleFix":"// before\nreq.stop_token_ids = Some(vec![151645, 999999]); // 999999 > vocab\n\n// after\nreq.stop_token_ids = Some(vec![151645]);\n// or express by text\nreq.stop = Some(vec![\"<|im_end|>\".into()]);","handlingStrategy":"validation","validationCode":"let vocab = server_vocab_size; // from model metadata\nfor (name, ids) in [(\"stop_token_ids\", &stop_token_ids), (\"allowed_token_ids\", &allowed_token_ids), (\"logprob_token_ids\", &logprob_token_ids)] {\n    assert!(ids.iter().all(|&t| (t as usize) < vocab),\n        \"{name} contains IDs >= vocab size {vocab}\");\n}","typeGuard":"fn is_out_of_vocab(e: &Error) -> bool {\n    matches!(e, Error::TokenIds(TokenIdsError::OutOfVocab { .. }))\n}","tryCatchPattern":"match err {\n    Error::TokenIds(TokenIdsError::OutOfVocab { parameter, token_ids, vocab_size }) =>\n        bad_request(format!(\"{parameter} {token_ids:?} >= vocab {vocab_size}; regenerate IDs with the served tokenizer\")),\n    _ => /* ... */\n}","preventionTips":["Never cache raw token IDs across model changes; re-tokenize per served model","Prefer string stop/bad-words fields and let the server tokenize"],"tags":["rust","validation","token-ids","tokenizer","vllm"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}