sgl-project/sglang · error · ValueError

draft_token_num must be positive, got {draft_token_num}.

Error message

draft_token_num must be positive, got {draft_token_num}.

What it means

Raised by apply_dflash_verify_logits_adjustments when the draft_token_num argument is zero or negative. DFlash speculative decoding applies per-draft-token logit adjustments during verify, so the number of draft tokens per sequence must be a positive integer. A non-positive value means the caller misconfigured the speculative draft length upstream.

Source

Thrown at python/sglang/srt/speculative/dflash_utils.py:227

    next_token_logits: torch.Tensor,
    sampling_info: Any,
    draft_token_num: int,
) -> None:
    """Apply sampling-time logit adjustments for DFlash verify in place.

    This keeps v1 and v2 verify semantics aligned while letting overlap scheduling
    use the cheaper precomputed `acc_linear_penalties` path instead of allocating a
    repeated `[bs * draft_token_num, vocab]` penalty tensor every step.
    """
    if sampling_info is None:
        return
    if next_token_logits.ndim != 2:
        raise ValueError(
            "next_token_logits must be 2D, "
            f"got shape={tuple(next_token_logits.shape)}."
        )
    if draft_token_num <= 0:
        raise ValueError(f"draft_token_num must be positive, got {draft_token_num}.")

    bs = len(sampling_info)
    if next_token_logits.shape[0] != bs * draft_token_num:
        raise ValueError(
            "next_token_logits row count mismatch for DFlash verify adjustments. "
            f"Expected {bs * draft_token_num}, got {next_token_logits.shape[0]}."
        )

    if sampling_info.has_custom_logit_processor:
        apply_custom_logit_processor(
            next_token_logits,
            sampling_info,
            num_tokens_in_batch=draft_token_num,
        )

    acc_linear_penalties = getattr(sampling_info, "acc_linear_penalties", None)
    penalizer = getattr(sampling_info, "penalizer_orchestrator", None)
    grammar_mask = getattr(sampling_info, "grammar_mask", None)

View on GitHub (pinned to 0132848349)

Solutions

  1. Set the draft token count to a positive value (e.g. --speculative-num-draft-tokens 8 or spec_num_draft_tokens in the draft config).
  2. Check the resolved DFlashDraftConfig / server args before starting the server to confirm the draft length propagated correctly.
  3. If constructing batches manually in tests, pass an explicit positive draft_token_num matching the logits rows.

Example fix

# before
apply_dflash_verify_logits_adjustments(logits, sampling_info, draft_token_num=0)
# after
apply_dflash_verify_logits_adjustments(logits, sampling_info, draft_token_num=8)
Defensive patterns

Strategy: validation

Validate before calling

if draft_token_num is None or draft_token_num <= 0:
    raise ValueError(f"draft_token_num must be positive, got {draft_token_num!r}")

Type guard

def is_positive_int(v) -> bool:
    return isinstance(v, int) and not isinstance(v, bool) and v > 0

Prevention

When it happens

Trigger: Calling apply_dflash_verify_logits_adjustments(next_token_logits, sampling_info, draft_token_num) with draft_token_num=0 or a negative value, e.g. server args --speculative-num-draft-tokens 0 or a draft config that resolved num_draft_tokens to 0.

Common situations: Server launched with speculative decoding enabled but speculative-num-draft-tokens set to 0; a draft config JSON with an invalid draft token count; tests constructing a DFlash verify batch with default/zeroed draft params.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9d6b3e7605c0ecb2. Report an issue: GitHub.