{"record":{"id":"aafc1cc3f0a18096","repo":"vllm-project/vllm","slug":"long-prefill-token-threshold-self-long-prefill-t","errorCode":null,"errorMessage":"long_prefill_token_threshold ({self.long_prefill_token_threshold}) cannot be greater than the max_model_len ({max_model_len}).","messagePattern":"long_prefill_token_threshold \\((.+?)\\) cannot be greater than the max_model_len \\((.+?)\\)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/scheduler.py","lineNumber":279,"sourceCode":"            )\n\n        if self.max_num_batched_tokens < self.max_num_seqs:\n            raise ValueError(\n                f\"max_num_batched_tokens ({self.max_num_batched_tokens}) must \"\n                \"be greater than or equal to max_num_seqs \"\n                f\"({self.max_num_seqs}).\"\n            )\n\n        if self.max_num_batched_tokens > self.max_num_seqs * max_model_len:\n            logger.warning(\n                \"max_num_batched_tokens (%d) exceeds max_num_seqs \"\n                \"* max_model_len (%d). This may lead to unexpected behavior.\",\n                self.max_num_batched_tokens,\n                self.max_num_seqs * max_model_len,\n            )\n\n        if self.long_prefill_token_threshold > max_model_len:\n            raise ValueError(\n                \"long_prefill_token_threshold \"\n                f\"({self.long_prefill_token_threshold}) cannot be greater \"\n                f\"than the max_model_len ({max_model_len}).\"\n            )\n\n        return self\n","sourceCodeStart":261,"sourceCodeEnd":286,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/scheduler.py#L261-L286","documentation":"verify_max_model_len caps long_prefill_token_threshold at max_model_len: the threshold decides when a prefill is treated as 'long', and a threshold beyond the longest possible sequence is meaningless and would misroute every prefill into the long path.","triggerScenarios":"Setting --long-prefill-token-threshold 65536 with --max-model-len 8192; keeping a tuned threshold after lowering max_model_len for memory; defaults from a benchmark script applied to a shorter-context model.","commonSituations":"Performance tuning carried over between models; shrinking context windows after threshold tuning; config files shared across heterogeneous deployments.","solutions":["Lower long_prefill_token_threshold to <= max_model_len (typically a fraction of it)","Or raise --max-model-len if the long context is actually needed","Add a startup assertion in deploy scripts: threshold <= max_model_len"],"exampleFix":"# before\n--max-model-len 8192 --long-prefill-token-threshold 32768\n\n# after\n--max-model-len 8192 --long-prefill-token-threshold 4096","handlingStrategy":"validation","validationCode":"def threshold_ok(threshold: int, max_model_len: int) -> bool:\n    return threshold <= max_model_len","typeGuard":"null","tryCatchPattern":"null","preventionTips":["Tie long_prefill_token_threshold to a fraction of max_model_len in scripts","Validate after lowering max_model_len"],"tags":["scheduler","config","validation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}