{"record":{"id":"2c83327a6916923b","repo":"vllm-project/vllm","slug":"vllmconfig-does-not-have-enough-slots-to-schedule","errorCode":null,"errorMessage":"VllmConfig does not have enough slots to schedule a token and support the speculative decoding settings. Got {max_num_batched_tokens=} and {scheduled_token_delta=}.","messagePattern":"VllmConfig does not have enough slots to schedule a token and support the speculative decoding settings\\. Got (.+?) and (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/vllm.py","lineNumber":1873,"sourceCode":"                    \"max_num_scheduled_tokens is set to\"\n                    f\" {self.scheduler_config.max_num_scheduled_tokens} based on\"\n                    \" the speculative decoding settings, which does not allow\"\n                    \" any tokens to be scheduled. Increase max_num_batched_tokens\"\n                    \" to accommodate the additional draft token slots, or decrease\"\n                    \" num_speculative_tokens.\"\n                )\n            if self.scheduler_config.max_num_scheduled_tokens < 8192:\n                logger.warning_once(\n                    \"max_num_scheduled_tokens is set to\"\n                    f\" {self.scheduler_config.max_num_scheduled_tokens} based on\"\n                    \" the speculative decoding settings. This may lead to suboptimal\"\n                    \" performance. Consider increasing max_num_batched_tokens to\"\n                    \" accommodate the additional draft token slots, or decrease\"\n                    \" num_speculative_tokens.\",\n                )\n\n            if max_num_batched_tokens <= scheduled_token_delta:\n                raise ValueError(\n                    \"VllmConfig does not have enough slots to schedule a token and\"\n                    \" support the speculative decoding settings.\"\n                    f\" Got {max_num_batched_tokens=} and {scheduled_token_delta=}.\"\n                )\n\n    def _set_cudagraph_sizes(self):\n        \"\"\"\n        vLLM defines the default candidate list of batch sizes for CUDA graph\n        capture as:\n\n        ```python\n        default_max_graph_size = 1024 if is_data_center_blackwell else 512\n        max_graph_size = min(max_num_seqs * decode_query_len * 2,\n                             default_max_graph_size)\n        # 1, 2, 4, then multiples of 8 up to 256 and then multiples of 16\n        # up to max_graph_size\n        cudagraph_capture_sizes = [1, 2, 4] + list(range(8, 256, 8)) + list(\n            range(256, max_graph_size + 1, 16))","sourceCodeStart":1855,"sourceCodeEnd":1891,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/vllm.py#L1855-L1891","documentation":"A stricter companion check to [391]: even when max_num_scheduled_tokens is positive, the config must leave at least one slot for a real token after reserving `scheduled_token_delta` draft slots. If `max_num_batched_tokens <= max_num_new_slots_for_drafting`, validation fails with both values printed.","triggerScenarios":"Any speculative-decode launch where `--max-num-batched-tokens` is less than or equal to `speculative_config.max_num_new_slots_for_drafting` (a function of num_speculative_tokens and lookahead/k values, computed in vllm/config/speculative.py:1421).","commonSituations":"Aggressive speculative settings (large num_speculative_tokens or ngram k) on top of a small max_num_batched_tokens; auto-derived batched tokens on memory-constrained GPUs.","solutions":["Raise `--max-num-batched-tokens` above the reported scheduled_token_delta value.","Or reduce `num_speculative_tokens` (or ngram k / lookahead) to shrink the reservation.","Leave max_num_batched_tokens at default unless memory forces a reduction."],"exampleFix":"# before (error reports e.g. max_num_batched_tokens=256, scheduled_token_delta=512)\nvllm serve model --max-num-batched-tokens 256 \\\n  --speculative-config '{\"method\":\"ngram\",\"num_speculative_tokens\":56,...}'\n\n# after\nvllm serve model --max-num-batched-tokens 4096 \\\n  --speculative-config '{\"method\":\"ngram\",\"num_speculative_tokens\":56,...}'","handlingStrategy":"validation","validationCode":"from vllm.config import SpeculativeConfig\n# before engine start\nsc = SpeculativeConfig(**spec_kwargs)\nif max_num_batched_tokens <= sc.max_num_new_slots_for_drafting:\n    raise SystemExit(f\"max_num_batched_tokens must be > {sc.max_num_new_slots_for_drafting}\")","typeGuard":null,"tryCatchPattern":"try:\n    LLM(**args)\nexcept ValueError as e:\n    if \"not enough slots to schedule\" in str(e):\n        args[\"max_num_batched_tokens\"] *= 4  # then retry\n    else:\n        raise","preventionTips":["Compute max_num_new_slots_for_drafting from the speculative config before choosing batched tokens","Prefer default max_num_batched_tokens unless memory-bound"],"tags":["scheduler","speculative-decoding","batched-tokens"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}