{"record":{"id":"3ca77c0599b73dfa","repo":"vllm-project/vllm","slug":"max-num-batched-tokens-self-max-num-batched-toke-3ca77c","errorCode":null,"errorMessage":"max_num_batched_tokens ({self.max_num_batched_tokens}) must be greater than or equal to max_num_seqs ({self.max_num_seqs}).","messagePattern":"max_num_batched_tokens \\((.+?)\\) must be greater than or equal to max_num_seqs \\((.+?)\\)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/scheduler.py","lineNumber":264,"sourceCode":"\n        self.verify_max_model_len(max_model_len)\n\n    def verify_max_model_len(self, max_model_len: int) -> Self:\n        if (\n            self.max_num_batched_tokens < max_model_len\n            and not self.enable_chunked_prefill\n        ):\n            raise ValueError(\n                f\"max_num_batched_tokens ({self.max_num_batched_tokens}) is \"\n                f\"smaller than max_model_len ({max_model_len}). \"\n                \"This effectively limits the maximum sequence length to \"\n                \"max_num_batched_tokens and makes vLLM reject longer \"\n                \"sequences. Please increase max_num_batched_tokens or \"\n                \"decrease max_model_len.\"\n            )\n\n        if self.max_num_batched_tokens < self.max_num_seqs:\n            raise ValueError(\n                f\"max_num_batched_tokens ({self.max_num_batched_tokens}) must \"\n                \"be greater than or equal to max_num_seqs \"\n                f\"({self.max_num_seqs}).\"\n            )\n\n        if self.max_num_batched_tokens > self.max_num_seqs * max_model_len:\n            logger.warning(\n                \"max_num_batched_tokens (%d) exceeds max_num_seqs \"\n                \"* max_model_len (%d). This may lead to unexpected behavior.\",\n                self.max_num_batched_tokens,\n                self.max_num_seqs * max_model_len,\n            )\n\n        if self.long_prefill_token_threshold > max_model_len:\n            raise ValueError(\n                \"long_prefill_token_threshold \"\n                f\"({self.long_prefill_token_threshold}) cannot be greater \"\n                f\"than the max_model_len ({max_model_len}).\"","sourceCodeStart":246,"sourceCodeEnd":282,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/scheduler.py#L246-L282","documentation":"verify_max_model_len also requires max_num_batched_tokens >= max_num_seqs: each running sequence must be able to receive at least one token per scheduler step within the batched-token budget. Otherwise some sequences would be starved indefinitely.","triggerScenarios":"--max-num-batched-tokens 64 with --max-num-seqs 256; deriving one of the two from GPU memory formulas independently so they cross; setting max_num_seqs high for throughput on small-token budgets.","commonSituations":"Aggressive concurrency tuning; copying configs between hardware profiles; defaults that assume a larger token budget than the operator set.","solutions":["Raise --max-num-batched-tokens to >= max_num_seqs","Lower --max-num-seqs to <= max_num_batched_tokens","Scale both together (e.g. budget = seqs * tokens-per-step) in deployment templates"],"exampleFix":"# before\n--max-num-batched-tokens 64 --max-num-seqs 256\n\n# after\n--max-num-batched-tokens 4096 --max-num-seqs 256","handlingStrategy":"validation","validationCode":"def seqs_fit_budget(mnbt: int, max_num_seqs: int) -> bool:\n    return mnbt >= max_num_seqs","typeGuard":"null","tryCatchPattern":"null","preventionTips":["Derive max_num_seqs from the token budget, never tune them independently","Add config-lint checks comparing the two values"],"tags":["scheduler","config","validation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}