vllm-project/vllm · error · ValueError

{speculative_max_model_len=} cannot be larger than {draft_ma

Error message

{speculative_max_model_len=} cannot be larger than {draft_max_model_len=}

What it means

Raised by the static helper _maybe_override_draft_max_model_len when the explicitly passed speculative_max_model_len exceeds draft_max_model_len (the draft model's max length). The draft model must be able to attend over every position the speculative run can reach, so a speculative window larger than the draft's capacity is rejected.

Source

Thrown at vllm/config/speculative.py:1202

        speculative_max_model_len: int | None,
        draft_max_model_len: int,
        target_max_model_len: int,
    ) -> int:
        """Determine the max sequence len for the draft model. This is usually
        the draft_max_model_len, but may be the target_max_model_len if it is
        less than the draft_max_model_len, or may be speculative_max_model_len
        if it is specified.

        This is necessary so that sequences do not exceed the capacity of the
        draft model or the target model.

        speculative_max_model_len is mainly used for testing that sequences can
        skip speculation.
        """

        if speculative_max_model_len is not None:
            if speculative_max_model_len > draft_max_model_len:
                raise ValueError(
                    f"{speculative_max_model_len=} cannot be "
                    f"larger than {draft_max_model_len=}"
                )

            if speculative_max_model_len > target_max_model_len:
                raise ValueError(
                    f"{speculative_max_model_len=} cannot be "
                    f"larger than {target_max_model_len=}"
                )

            return speculative_max_model_len

        result = min(
            draft_max_model_len,
            target_max_model_len,
        )
        if result != draft_max_model_len:
            logger.info(

View on GitHub (pinned to c794754062)

Solutions

1. Set speculative_max_model_len <= the draft model's max_model_len
2. Omit speculative_max_model_len so it defaults to min(draft_max_model_len, target_max_model_len)
3. Use a draft checkpoint whose context covers the desired speculative length

Example fix

# before
speculative_config={"method": "eagle", "model": "draft32k", "num_speculative_tokens": 3, "speculative_max_model_len": 65536}
# after
speculative_config={"method": "eagle", "model": "draft32k", "num_speculative_tokens": 3, "speculative_max_model_len": 32768}

from transformers import AutoConfig
draft_len = AutoConfig.from_pretrained(draft_model).max_position_embeddings
if (sml := spec_cfg.get("speculative_max_model_len")) and sml > draft_len:
    spec_cfg["speculative_max_model_len"] = min(sml, draft_len)  # or drop the key

def fits_draft_capacity(spec_max_len: int | None, draft_max_len: int) -> bool:
    return spec_max_len is None or spec_max_len <= draft_max_len

When it happens

Trigger: Passing speculative_max_model_len larger than the draft model's max_model_len, e.g. target 128k, draft 32k, speculative_max_model_len=65536. Usually via the speculative_config dict for testing speculation-skip behavior.

Common situations: Using a short-context EAGLE draft with a long-context target and forcing a large speculative length; test harnesses that set speculative_max_model_len to the target's context without checking the draft.

Related errors

• {speculative_max_model_len=} cannot be larger than {target_m
• rejection_sample_method='synthetic' requires exactly one of
• synthetic_acceptance_rates must have length {n}, got {rates}
• synthetic_acceptance_rates entries must be in [0, 1], got {r
• synthetic_acceptance_rates must be non-increasing, got {rate

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/7a3a63b2c8c404d9. Report an issue: GitHub.