{"record":{"id":"2238a9c19b4dcad0","repo":"vllm-project/vllm","slug":"this-model-s-maximum-context-length-is-max-model-2238a9","errorCode":null,"errorMessage":"this model's maximum context length is {max_model_len} tokens, but the prompt contains {prompt_len} input tokens","messagePattern":"this model's maximum context length is (.+?) tokens, but the prompt contains (.+?) input tokens","errorType":"validation","errorClass":"Error","httpStatus":400,"severity":"error","filePath":"rust/src/text/src/error.rs","lineNumber":20,"sourceCode":"// SPDX-FileCopyrightText: Copyright contributors to the vLLM project\n\nuse thiserror::Error;\nuse vllm_engine_core_client::Error as EngineCoreError;\nuse vllm_llm::Error as LlmError;\n\npub use crate::lower::logprobs::LogprobsError;\npub use crate::lower::sampling::SamplingParamsError;\npub use crate::lower::token_ids::TokenIdsError;\n\n#[derive(Debug, Error)]\npub enum Error {\n    #[error(\"tokenizer error: {0}\")]\n    Tokenizer(String),\n    #[error(\"text request `{request_id}` must contain at least one prompt token ID\")]\n    EmptyPromptTokenIds { request_id: String },\n    #[error(\"text request `{request_id}` stop strings cannot be empty\")]\n    EmptyStopString { request_id: String },\n    #[error(\n        \"this model's maximum context length is {max_model_len} tokens, \\\n         but the prompt contains {prompt_len} input tokens\"\n    )]\n    PromptTooLong { max_model_len: u32, prompt_len: u32 },\n    #[error(transparent)]\n    Logprobs(#[from] LogprobsError),\n    #[error(transparent)]\n    TokenIds(#[from] TokenIdsError),\n    #[error(transparent)]\n    SamplingParams(#[from] SamplingParamsError),\n    #[error(\n        \"`min_tokens` must be less than or equal to `max_tokens`, \\\n         got min_tokens={min_tokens}, max_tokens={max_tokens}\"\n    )]\n    MinTokensExceedsMaxTokens { min_tokens: u32, max_tokens: u32 },\n    #[error(\"`thinking_token_budget` must be a non-negative integer or -1 for unlimited.\")]\n    InvalidThinkingTokenBudget,\n    #[error(\"invalid repetition detection params: {message}\")]","sourceCodeStart":2,"sourceCodeEnd":38,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/rust/src/text/src/error.rs#L2-L38","documentation":"Error variant raised in rust/src/text/src/lower.rs (three sites: 290, 1334, 1346) when the tokenized prompt length exceeds the model's maximum context length (max_model_len). The message carries both limits: 'this model's maximum context length is {max_model_len} tokens, but the prompt contains {prompt_len} input tokens'. Classified by is_request_validation_error() as a client error, matching OpenAI-compatible behavior of returning 400 for oversized prompts.","triggerScenarios":"POST /v1/completions or /v1/chat/completions whose prompt/messages tokenize to more tokens than the model's context window — e.g. a 200k-token document to an 8k model, or a chat history that grew across turns. Also triggered via prompt_token_ids whose length exceeds max_model_len.","commonSituations":"RAG pipelines stuffing whole documents into the prompt; multi-turn chat servers accumulating history without truncation; mixing up model variants (128k vs 4k) after switching --model; counting characters instead of tokens when estimating size.","solutions":["Truncate the prompt/messages client-side to fit (remember to leave room for max_new_tokens).","Switch to a longer-context model or raise --max-model-len if the model weights support it.","Tokenize with the same tokenizer and check len(tokens) + max_tokens <= max_model_len before sending.","For chat, prune old messages or summarize history to keep it under the window."],"exampleFix":"# before\n{\"model\": \"8k-model\", \"prompt\": \"<200k tokens of text>\"}\n\n# after\n# client-side guard (Python):\n# ids = tok.encode(prompt)\n# if len(ids) > max_model_len - max_new_tokens: prompt = tok.decode(ids[:limit])\n{\"model\": \"128k-model\", \"prompt\": \"<same text>\", \"max_tokens\": 512}","handlingStrategy":"validation","validationCode":"let prompt_len: u32 = tokenizer.encode(prompt).len() as u32;\nif prompt_len + max_new_tokens > max_model_len {\n    return Err(format!(\"prompt {prompt_len} + {max_new_tokens} tokens exceeds {max_model_len}\"));\n}","typeGuard":"fn prompt_fits(prompt_len: u32, max_new_tokens: u32, max_model_len: u32) -> bool {\n    prompt_len.saturating_add(max_new_tokens) <= max_model_len\n}","tryCatchPattern":"match result {\n    Err(e @ vllm_text::Error::PromptTooLong { max_model_len, prompt_len }) => {\n        // client error: truncate input or reroute to a longer-context model; do not retry unchanged\n        respond_400_with_hint(e, format!(\"trim ~{} tokens\", prompt_len - max_model_len + 1));\n    }\n    other => other,\n}","preventionTips":["Tokenize and length-check every prompt with the model's own tokenizer before sending.","Budget for output: keep prompt_len + max_tokens <= max_model_len.","For chat, cap history size or summarize older turns; never estimate size by characters."],"tags":["request-validation","context-length","prompt","tokenizer","api","rust","vllm"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}