{"record":{"id":"7a3a63b2c8c404d9","repo":"vllm-project/vllm","slug":"speculative-max-model-len-cannot-be-larger-than","errorCode":null,"errorMessage":"{speculative_max_model_len=} cannot be larger than {draft_max_model_len=}","messagePattern":"(.+?) cannot be larger than (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/speculative.py","lineNumber":1202,"sourceCode":"        speculative_max_model_len: int | None,\n        draft_max_model_len: int,\n        target_max_model_len: int,\n    ) -> int:\n        \"\"\"Determine the max sequence len for the draft model. This is usually\n        the draft_max_model_len, but may be the target_max_model_len if it is\n        less than the draft_max_model_len, or may be speculative_max_model_len\n        if it is specified.\n\n        This is necessary so that sequences do not exceed the capacity of the\n        draft model or the target model.\n\n        speculative_max_model_len is mainly used for testing that sequences can\n        skip speculation.\n        \"\"\"\n\n        if speculative_max_model_len is not None:\n            if speculative_max_model_len > draft_max_model_len:\n                raise ValueError(\n                    f\"{speculative_max_model_len=} cannot be \"\n                    f\"larger than {draft_max_model_len=}\"\n                )\n\n            if speculative_max_model_len > target_max_model_len:\n                raise ValueError(\n                    f\"{speculative_max_model_len=} cannot be \"\n                    f\"larger than {target_max_model_len=}\"\n                )\n\n            return speculative_max_model_len\n\n        result = min(\n            draft_max_model_len,\n            target_max_model_len,\n        )\n        if result != draft_max_model_len:\n            logger.info(","sourceCodeStart":1184,"sourceCodeEnd":1220,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/speculative.py#L1184-L1220","documentation":"Raised by the static helper _maybe_override_draft_max_model_len when the explicitly passed speculative_max_model_len exceeds draft_max_model_len (the draft model's max length). The draft model must be able to attend over every position the speculative run can reach, so a speculative window larger than the draft's capacity is rejected.","triggerScenarios":"Passing speculative_max_model_len larger than the draft model's max_model_len, e.g. target 128k, draft 32k, speculative_max_model_len=65536. Usually via the speculative_config dict for testing speculation-skip behavior.","commonSituations":"Using a short-context EAGLE draft with a long-context target and forcing a large speculative length; test harnesses that set speculative_max_model_len to the target's context without checking the draft.","solutions":["Set speculative_max_model_len <= the draft model's max_model_len","Omit speculative_max_model_len so it defaults to min(draft_max_model_len, target_max_model_len)","Use a draft checkpoint whose context covers the desired speculative length"],"exampleFix":"# before\nspeculative_config={\"method\": \"eagle\", \"model\": \"draft32k\", \"num_speculative_tokens\": 3, \"speculative_max_model_len\": 65536}\n# after\nspeculative_config={\"method\": \"eagle\", \"model\": \"draft32k\", \"num_speculative_tokens\": 3, \"speculative_max_model_len\": 32768}","handlingStrategy":"validation","validationCode":"from transformers import AutoConfig\ndraft_len = AutoConfig.from_pretrained(draft_model).max_position_embeddings\nif (sml := spec_cfg.get(\"speculative_max_model_len\")) and sml > draft_len:\n    spec_cfg[\"speculative_max_model_len\"] = min(sml, draft_len)  # or drop the key","typeGuard":"def fits_draft_capacity(spec_max_len: int | None, draft_max_len: int) -> bool:\n    return spec_max_len is None or spec_max_len <= draft_max_len","tryCatchPattern":null,"preventionTips":["Prefer omitting speculative_max_model_len (it defaults to min(draft, target)) unless testing speculation-skip","Compare any explicit speculative length against both models' config.json before launching"],"tags":["speculative-decoding","max-model-len","draft-model","validation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}