vllm-project/vllm · error · ValueError

{speculative_max_model_len=} cannot be larger than {target_m

Error message

{speculative_max_model_len=} cannot be larger than {target_max_model_len=}

What it means

Raised by _maybe_override_draft_max_model_len when speculative_max_model_len exceeds target_max_model_len. Even if the draft could handle it, the target model cannot attend beyond its own max_model_len, so the speculative window is clamped-and-validated against the target as well.

Source

Thrown at vllm/config/speculative.py:1208

        less than the draft_max_model_len, or may be speculative_max_model_len
        if it is specified.

        This is necessary so that sequences do not exceed the capacity of the
        draft model or the target model.

        speculative_max_model_len is mainly used for testing that sequences can
        skip speculation.
        """

        if speculative_max_model_len is not None:
            if speculative_max_model_len > draft_max_model_len:
                raise ValueError(
                    f"{speculative_max_model_len=} cannot be "
                    f"larger than {draft_max_model_len=}"
                )

            if speculative_max_model_len > target_max_model_len:
                raise ValueError(
                    f"{speculative_max_model_len=} cannot be "
                    f"larger than {target_max_model_len=}"
                )

            return speculative_max_model_len

        result = min(
            draft_max_model_len,
            target_max_model_len,
        )
        if result != draft_max_model_len:
            logger.info(
                "Overriding draft model max model len from %d to %d",
                draft_max_model_len,
                result,
            )
        return result

View on GitHub (pinned to c794754062)

Solutions

Set speculative_max_model_len <= the target's max_model_len
Raise --max-model-len on the target if a longer window is genuinely needed (verify GPU memory allows it)
Omit speculative_max_model_len to use the automatic min(draft, target) default

Example fix

# before
llm = LLM(model=target, max_model_len=65536, speculative_config={"method": "eagle", "speculative_max_model_len": 100000, ...})
# after
llm = LLM(model=target, max_model_len=65536, speculative_config={"method": "eagle", "speculative_max_model_len": 65536, ...})

Defensive patterns

Strategy: validation

Validate before calling

if (sml := spec_cfg.get("speculative_max_model_len")) and sml > target_max_model_len:
    raise ValueError("raise --max-model-len or lower speculative_max_model_len")

Type guard

def fits_target_capacity(spec_max_len: int | None, target_max_len: int) -> bool:
    return spec_max_len is None or spec_max_len <= target_max_len

Prevention

Derive speculative_max_model_len from the effective --max-model-len, not the checkpoint's trained context
Re-run config checks whenever max_model_len is tuned down for memory

When it happens

Trigger: Passing speculative_max_model_len greater than the target's --max-model-len (e.g. 100000 against a 65536 target) while also having a draft with sufficient or larger capacity.

Common situations: Raising speculative_max_model_len without raising --max-model-len; deriving the value from the checkpoint's trained context while the server caps max_model_len lower for memory.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/c1296c305fd63ad1. Report an issue: GitHub.