{"record":{"id":"ed187b46b5afb7f3","repo":"vllm-project/vllm","slug":"max-num-batched-tokens-self-max-num-batched-toke","errorCode":null,"errorMessage":"max_num_batched_tokens ({self.max_num_batched_tokens}) is smaller than max_model_len ({max_model_len}). This effectively limits the maximum sequence length to max_num_batched_tokens and makes vLLM reject longer sequences. Please increase max_num_batched_tokens or decrease max_model_len.","messagePattern":"max_num_batched_tokens \\((.+?)\\) is smaller than max_model_len \\((.+?)\\)\\. This effectively limits the maximum sequence length to max_num_batched_tokens and makes vLLM reject longer sequences\\. Please increase max_num_batched_tokens or decrease max_model_len\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/scheduler.py","lineNumber":254,"sourceCode":"            )\n\n        self.max_num_encoder_input_tokens = self.max_num_batched_tokens\n        self.encoder_cache_size = self.max_num_batched_tokens\n\n        if self.enable_chunked_prefill:\n            logger.info_once(\n                \"Chunked prefill is enabled with max_num_batched_tokens=%d.\",\n                self.max_num_batched_tokens,\n            )\n\n        self.verify_max_model_len(max_model_len)\n\n    def verify_max_model_len(self, max_model_len: int) -> Self:\n        if (\n            self.max_num_batched_tokens < max_model_len\n            and not self.enable_chunked_prefill\n        ):\n            raise ValueError(\n                f\"max_num_batched_tokens ({self.max_num_batched_tokens}) is \"\n                f\"smaller than max_model_len ({max_model_len}). \"\n                \"This effectively limits the maximum sequence length to \"\n                \"max_num_batched_tokens and makes vLLM reject longer \"\n                \"sequences. Please increase max_num_batched_tokens or \"\n                \"decrease max_model_len.\"\n            )\n\n        if self.max_num_batched_tokens < self.max_num_seqs:\n            raise ValueError(\n                f\"max_num_batched_tokens ({self.max_num_batched_tokens}) must \"\n                \"be greater than or equal to max_num_seqs \"\n                f\"({self.max_num_seqs}).\"\n            )\n\n        if self.max_num_batched_tokens > self.max_num_seqs * max_model_len:\n            logger.warning(\n                \"max_num_batched_tokens (%d) exceeds max_num_seqs \"","sourceCodeStart":236,"sourceCodeEnd":272,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/scheduler.py#L236-L272","documentation":"SchedulerConfig.verify_max_model_len rejects configs where max_num_batched_tokens < max_model_len and chunked prefill is off. Without chunking, a single prefill must fit in one batched-tokens budget, so any prompt longer than max_num_batched_tokens could never be scheduled.","triggerScenarios":"Small --max-num-batched-tokens (e.g. 2048) with a model whose max_model_len is 8192+ and --enable-chunked-prefill absent/false; deriving max_num_batched_tokens from a default while loading a long-context model; API server startup with explicit small budget.","commonSituations":"Tuning batched tokens down for memory on GPUs; enabling long-context models (32k/128k) while keeping old scheduler settings; chunked prefill disabled for cudagraph/compat reasons on older versions.","solutions":["Enable chunked prefill (--enable-chunked-prefill) so long prompts are split across steps","Raise --max-num-batched-tokens to at least max_model_len","Lower --max-model-len to fit within the current budget","Set a VLLM_* default/env or config that keeps the two consistent at deploy time"],"exampleFix":"# before\n--max-num-batched-tokens 2048 --max-model-len 32768\n\n# after\n--max-num-batched-tokens 2048 --max-model-len 32768 --enable-chunked-prefill","handlingStrategy":"validation","validationCode":"def scheduler_ok(mnbt: int, max_model_len: int, chunked: bool) -> bool:\n    return chunked or mnbt >= max_model_len","typeGuard":"null","tryCatchPattern":"null","preventionTips":["Default to --enable-chunked-prefill on long-context models","Assert mnbt >= max_model_len (or chunked) in deploy templates","Re-check after any max-model-len change"],"tags":["scheduler","config","context-length","validation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}