sgl-project/sglang · error · ValueError

DSpark speculative_num_draft_tokens must be >= 2 (= gamma +

Error message

DSpark speculative_num_draft_tokens must be >= 2 (= gamma + 1), got {num_draft_tokens}.

What it means

DSpark computes gamma (number of draft tokens per step) as speculative_num_draft_tokens - 1, so num_draft_tokens must be at least 2. Passing 1 or 0 makes gamma < 1 which is nonsensical for speculative proposal, and dspark_gamma_from_num_draft_tokens raises immediately.

Source

Thrown at python/sglang/srt/speculative/dspark_components/dspark_config.py:53

    from sglang.srt.utils.hf_transformers_utils import get_config

    draft_model_path = get_spec().speculative_draft_model_path
    if not draft_model_path:
        return False
    draft_hf_config = get_config(
        draft_model_path,
        trust_remote_code=get_model().trust_remote_code,
        revision=get_spec().speculative_draft_model_revision,
        model_override_args=json.loads(get_model().json_model_override_args),
        model_config_parser=get_model().model_config_parser,
    )
    return draft_hf_config is not None and is_deepseek_v4(draft_hf_config)


def dspark_gamma_from_num_draft_tokens(num_draft_tokens: int) -> int:
    gamma = int(num_draft_tokens) - 1
    if gamma < 1:
        raise ValueError(
            "DSpark speculative_num_draft_tokens must be >= 2 (= gamma + 1), "
            f"got {num_draft_tokens}."
        )
    return gamma


class DSparkDraftConfig(msgspec.Struct, frozen=True):
    num_hidden_layers: Optional[int]
    num_target_layers: Optional[int]
    gamma: Optional[int]
    target_layer_ids: Optional[List[int]]
    mask_token: str
    mask_token_id: Optional[int]
    markov_rank: int
    markov_head_type: Optional[str]

    def resolve_gamma(self, *, default: Optional[int] = None) -> Optional[int]:
        return self.gamma if self.gamma is not None else default

View on GitHub (pinned to 0132848349)

Solutions

1. Set --speculative-num-draft-tokens to >= 2 (gamma + 1), e.g. 2-8 depending on workload
2. Leave speculative_num_draft_tokens unset so gamma is resolved from the draft checkpoint config
3. Check the draft checkpoint's block_size/gamma default before overriding

Example fix

# before
--speculative-num-draft-tokens 1
# after
--speculative-num-draft-tokens 4

assert speculative_num_draft_tokens is None or speculative_num_draft_tokens >= 2, "num_draft_tokens must be >= 2 (gamma+1)"

When it happens

Trigger: Launching with --speculative-num-draft-tokens 1 (or 0) together with the DSpark speculative algorithm; resolve_runtime_config calls dspark_gamma_from_num_draft_tokens during worker init.

Common situations: Tuning speculative parameters and setting draft tokens to 1 thinking it disables drafting; misreading gamma vs num_draft_tokens semantics; config templates with a stale value.

Related errors

• Kimi-K3 DCP + DSPARK currently requires SGLANG_RAGGED_VERIFY
• BS {key}: candidate_steps must be a list of non-negative int
• speculative_adaptive_config must contain at least one intege
• DFLASH mask_token must be a non-empty string, got {mask_toke
• Invalid fused KV rotary/head dim pair: rotary_dim={rotary_di

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6e6030dc2603758f. Report an issue: GitHub.