{"record":{"id":"c1296c305fd63ad1","repo":"vllm-project/vllm","slug":"speculative-max-model-len-cannot-be-larger-than-c1296c","errorCode":null,"errorMessage":"{speculative_max_model_len=} cannot be larger than {target_max_model_len=}","messagePattern":"(.+?) cannot be larger than (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/speculative.py","lineNumber":1208,"sourceCode":"        less than the draft_max_model_len, or may be speculative_max_model_len\n        if it is specified.\n\n        This is necessary so that sequences do not exceed the capacity of the\n        draft model or the target model.\n\n        speculative_max_model_len is mainly used for testing that sequences can\n        skip speculation.\n        \"\"\"\n\n        if speculative_max_model_len is not None:\n            if speculative_max_model_len > draft_max_model_len:\n                raise ValueError(\n                    f\"{speculative_max_model_len=} cannot be \"\n                    f\"larger than {draft_max_model_len=}\"\n                )\n\n            if speculative_max_model_len > target_max_model_len:\n                raise ValueError(\n                    f\"{speculative_max_model_len=} cannot be \"\n                    f\"larger than {target_max_model_len=}\"\n                )\n\n            return speculative_max_model_len\n\n        result = min(\n            draft_max_model_len,\n            target_max_model_len,\n        )\n        if result != draft_max_model_len:\n            logger.info(\n                \"Overriding draft model max model len from %d to %d\",\n                draft_max_model_len,\n                result,\n            )\n        return result\n","sourceCodeStart":1190,"sourceCodeEnd":1226,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/speculative.py#L1190-L1226","documentation":"Raised by _maybe_override_draft_max_model_len when speculative_max_model_len exceeds target_max_model_len. Even if the draft could handle it, the target model cannot attend beyond its own max_model_len, so the speculative window is clamped-and-validated against the target as well.","triggerScenarios":"Passing speculative_max_model_len greater than the target's --max-model-len (e.g. 100000 against a 65536 target) while also having a draft with sufficient or larger capacity.","commonSituations":"Raising speculative_max_model_len without raising --max-model-len; deriving the value from the checkpoint's trained context while the server caps max_model_len lower for memory.","solutions":["Set speculative_max_model_len <= the target's max_model_len","Raise --max-model-len on the target if a longer window is genuinely needed (verify GPU memory allows it)","Omit speculative_max_model_len to use the automatic min(draft, target) default"],"exampleFix":"# before\nllm = LLM(model=target, max_model_len=65536, speculative_config={\"method\": \"eagle\", \"speculative_max_model_len\": 100000, ...})\n# after\nllm = LLM(model=target, max_model_len=65536, speculative_config={\"method\": \"eagle\", \"speculative_max_model_len\": 65536, ...})","handlingStrategy":"validation","validationCode":"if (sml := spec_cfg.get(\"speculative_max_model_len\")) and sml > target_max_model_len:\n    raise ValueError(\"raise --max-model-len or lower speculative_max_model_len\")","typeGuard":"def fits_target_capacity(spec_max_len: int | None, target_max_len: int) -> bool:\n    return spec_max_len is None or spec_max_len <= target_max_len","tryCatchPattern":null,"preventionTips":["Derive speculative_max_model_len from the effective --max-model-len, not the checkpoint's trained context","Re-run config checks whenever max_model_len is tuned down for memory"],"tags":["speculative-decoding","max-model-len","target-model","validation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}