sgl-project/sglang · error · ValueError

DFlash layer selection requires num_target_layers >= 4. Got

Error message

DFlash layer selection requires num_target_layers >= 4. Got num_target_layers={num_target_layers}.

What it means

Raised by build_target_layer_ids when num_target_layers < 4 and more than one draft layer is requested. The multi-draft-layer placement picks layers in the range [1, num_target_layers-3], which is empty for tiny models. Single-draft-layer configs bypass this (they use the midpoint), so only multi-layer drafts on very small models hit it.

Source

Thrown at python/sglang/srt/speculative/dflash_utils.py:383

    Notes:
        - DFlash uses hidden states after each selected target layer (HF-style).
        - SGLang captures "before layer i", so the model hook will typically add +1
          when mapping to capture points.
    """
    if num_target_layers <= 0:
        raise ValueError(
            f"num_target_layers must be positive, got {num_target_layers}."
        )
    if num_draft_layers <= 0:
        raise ValueError(f"num_draft_layers must be positive, got {num_draft_layers}.")

    if num_draft_layers == 1:
        return [num_target_layers // 2]

    start = 1
    end = num_target_layers - 3
    if end < start:
        raise ValueError(
            "DFlash layer selection requires num_target_layers >= 4. "
            f"Got num_target_layers={num_target_layers}."
        )

    span = end - start
    return [
        int(round(start + (i * span) / (num_draft_layers - 1)))
        for i in range(num_draft_layers)
    ]


def get_dflash_layer_types(config: Any) -> Optional[Sequence[str]]:
    text_config = _get_text_config(config)
    layer_types = _cfg_get(text_config, "layer_types", _cfg_get(config, "layer_types"))
    if layer_types is None:
        return None
    if isinstance(layer_types, str) or not isinstance(layer_types, Sequence):
        raise ValueError(

View on GitHub (pinned to 0132848349)

Solutions

1. Use num_draft_layers=1 for tiny target models (it falls back to the midpoint layer).
2. Use a target model with at least 4 hidden layers for multi-layer drafts.
3. Double-check you passed num_hidden_layers (a count like 32), not a layer index or per-block count.

Example fix

# before
build_target_layer_ids(num_target_layers=3, num_draft_layers=2)
# after
build_target_layer_ids(num_target_layers=3, num_draft_layers=1)

if num_draft_layers > 1 and num_target_layers < 4:
    num_draft_layers = 1  # fall back to single midpoint layer
# or reject early:
if num_draft_layers > 1 and num_target_layers < 4:
    raise ValueError('multi-layer DFlash draft needs >= 4 target layers')

When it happens

Trigger: Calling build_target_layer_ids with num_target_layers in {1,2,3} (end = num_target_layers-3 < start=1) and num_draft_layers >= 2, e.g. a 2-layer toy target model with a 2-layer draft.

Common situations: Running DFlash speculative decoding on tiny test/debug models (1-3 layers); unit tests with miniature configs requesting multiple draft layers; misreading num_hidden_layers (e.g. passing layer index instead of count).

Related errors

• num_target_layers must be positive, got {num_target_layers}.
• num_draft_layers must be positive, got {num_draft_layers}.
• {field_name} must be {comparator}, got {parsed}.
• DFLASH mask_token must be a non-empty string, got {mask_toke
• Invalid fused KV rotary/head dim pair: rotary_dim={rotary_di

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3cfbcc7249ab7f07. Report an issue: GitHub.