vllm-project/vllm · error · ValueError

dspark_draft_topk must be between 1 and the draft vocabulary

Error message

dspark_draft_topk must be between 1 and the draft vocabulary size ({draft_vocab_size})

What it means

Raised inside the dspark branch when the resolved dspark_draft_topk (from the CLI/config or the draft hf_config) falls outside [1, draft_vocab_size], where draft_vocab_size is hf_config.draft_vocab_size if present, else hf_config.vocab_size. The top-k logits slice must reference real vocabulary rows, so out-of-range k is rejected before the draft worker is built.

Source

Thrown at vllm/config/speculative.py:1106

                if self.dspark_draft_topk is not None and self.method != "dspark":
                    raise ValueError("dspark_draft_topk is only supported by DSpark")

                dspark_draft_topk = None
                if self.method == "dspark":
                    hf_config = self.draft_model_config.hf_config
                    dspark_draft_topk = self.dspark_draft_topk
                    if dspark_draft_topk is None:
                        dspark_draft_topk = getattr(
                            hf_config, "dspark_draft_topk", None
                        )
                    if dspark_draft_topk is not None:
                        draft_vocab_size = (
                            getattr(hf_config, "draft_vocab_size", None)
                            or hf_config.vocab_size
                        )
                        if not 1 <= dspark_draft_topk <= draft_vocab_size:
                            raise ValueError(
                                "dspark_draft_topk must be between 1 and the "
                                f"draft vocabulary size ({draft_vocab_size})"
                            )
                        if (
                            "Qwen3DSparkModel"
                            not in self.draft_model_config.architectures
                        ):
                            raise ValueError(
                                "dspark_draft_topk is only supported by "
                                "Qwen3DSparkModel"
                            )
                        hf_config.dspark_draft_topk = dspark_draft_topk

                self.draft_tensor_parallel_size = (
                    SpeculativeConfig._verify_and_get_draft_tp(
                        self.target_parallel_config,
                        self.draft_tensor_parallel_size,
                        self.draft_model_config.hf_config,

View on GitHub (pinned to c794754062)

Solutions

  1. Set dspark_draft_topk to a value in [1, draft_vocab_size]; check the draft config's vocab_size/draft_vocab_size first
  2. If the bad value comes from the checkpoint's hf_config, override it explicitly in speculative_config
  3. Lower the value to a conventional small k (e.g. 1-10) which is the intended usage]

Example fix

# before
speculative_config={"method": "dspark", "model": "...", "dspark_draft_topk": 200000}  # draft vocab 152k
# after
speculative_config={"method": "dspark", "model": "...", "dspark_draft_topk": 8}
Defensive patterns

Strategy: validation

Validate before calling

draft_vocab = getattr(hf_config, "draft_vocab_size", None) or hf_config.vocab_size
if not 1 <= topk <= draft_vocab:
    raise ValueError(f"dspark_draft_topk={topk} outside [1, {draft_vocab}]")

Type guard

def is_valid_dspark_topk(topk: int, draft_vocab_size: int) -> bool:
    return 1 <= topk <= draft_vocab_size

Prevention

When it happens

Trigger: Setting dspark_draft_topk to 0, a negative number, or a value larger than the draft model's vocab (e.g. 200000 against a 152k vocab draft); or a draft checkpoint whose embedded dspark_draft_topk exceeds its own vocab_size after tokenizer remapping.

Common situations: Tuning top-k for acceptance-rate vs. cost without checking the draft head size; mixing a large-vocab top-k value copied from a different model family.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/d994c654dc5bf15c. Report an issue: GitHub.