vllm-project/vllm · error · Error

this model's maximum context length is {max_model_len} token

Error message

this model's maximum context length is {max_model_len} tokens, but the prompt contains {prompt_len} input tokens

What it means

Error variant raised in rust/src/text/src/lower.rs (three sites: 290, 1334, 1346) when the tokenized prompt length exceeds the model's maximum context length (max_model_len). The message carries both limits: 'this model's maximum context length is {max_model_len} tokens, but the prompt contains {prompt_len} input tokens'. Classified by is_request_validation_error() as a client error, matching OpenAI-compatible behavior of returning 400 for oversized prompts.

Source

Thrown at rust/src/text/src/error.rs:20

// SPDX-FileCopyrightText: Copyright contributors to the vLLM project

use thiserror::Error;
use vllm_engine_core_client::Error as EngineCoreError;
use vllm_llm::Error as LlmError;

pub use crate::lower::logprobs::LogprobsError;
pub use crate::lower::sampling::SamplingParamsError;
pub use crate::lower::token_ids::TokenIdsError;

#[derive(Debug, Error)]
pub enum Error {
    #[error("tokenizer error: {0}")]
    Tokenizer(String),
    #[error("text request `{request_id}` must contain at least one prompt token ID")]
    EmptyPromptTokenIds { request_id: String },
    #[error("text request `{request_id}` stop strings cannot be empty")]
    EmptyStopString { request_id: String },
    #[error(
        "this model's maximum context length is {max_model_len} tokens, \
         but the prompt contains {prompt_len} input tokens"
    )]
    PromptTooLong { max_model_len: u32, prompt_len: u32 },
    #[error(transparent)]
    Logprobs(#[from] LogprobsError),
    #[error(transparent)]
    TokenIds(#[from] TokenIdsError),
    #[error(transparent)]
    SamplingParams(#[from] SamplingParamsError),
    #[error(
        "`min_tokens` must be less than or equal to `max_tokens`, \
         got min_tokens={min_tokens}, max_tokens={max_tokens}"
    )]
    MinTokensExceedsMaxTokens { min_tokens: u32, max_tokens: u32 },
    #[error("`thinking_token_budget` must be a non-negative integer or -1 for unlimited.")]
    InvalidThinkingTokenBudget,
    #[error("invalid repetition detection params: {message}")]

View on GitHub (pinned to c794754062)

Solutions

1. Truncate the prompt/messages client-side to fit (remember to leave room for max_new_tokens).
2. Switch to a longer-context model or raise --max-model-len if the model weights support it.
3. Tokenize with the same tokenizer and check len(tokens) + max_tokens <= max_model_len before sending.
4. For chat, prune old messages or summarize history to keep it under the window.

Example fix

# before
{"model": "8k-model", "prompt": "<200k tokens of text>"}

# after
# client-side guard (Python):
# ids = tok.encode(prompt)
# if len(ids) > max_model_len - max_new_tokens: prompt = tok.decode(ids[:limit])
{"model": "128k-model", "prompt": "<same text>", "max_tokens": 512}

let prompt_len: u32 = tokenizer.encode(prompt).len() as u32;
if prompt_len + max_new_tokens > max_model_len {
    return Err(format!("prompt {prompt_len} + {max_new_tokens} tokens exceeds {max_model_len}"));
}

fn prompt_fits(prompt_len: u32, max_new_tokens: u32, max_model_len: u32) -> bool {
    prompt_len.saturating_add(max_new_tokens) <= max_model_len
}

match result {
    Err(e @ vllm_text::Error::PromptTooLong { max_model_len, prompt_len }) => {
        // client error: truncate input or reroute to a longer-context model; do not retry unchanged
        respond_400_with_hint(e, format!("trim ~{} tokens", prompt_len - max_model_len + 1));
    }
    other => other,
}

When it happens

Trigger: POST /v1/completions or /v1/chat/completions whose prompt/messages tokenize to more tokens than the model's context window — e.g. a 200k-token document to an 8k model, or a chat history that grew across turns. Also triggered via prompt_token_ids whose length exceeds max_model_len.

Common situations: RAG pipelines stuffing whole documents into the prompt; multi-turn chat servers accumulating history without truncation; mixing up model variants (128k vs 4k) after switching --model; counting characters instead of tokens when estimating size.

Related errors

• text request `{request_id}` must contain at least one prompt
• text request `{request_id}` stop strings cannot be empty
• this model's maximum context length is {max_model_len} token
• tokenizer error: {0}
• token_id(s) {token_ids:?} in {parameter} are out of vocabula

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/2238a9c19b4dcad0. Report an issue: GitHub.