sgl-project/sglang · error · ValueError

num_target_layers must be positive, got {num_target_layers}.

Error message

num_target_layers must be positive, got {num_target_layers}.

What it means

Raised by build_target_layer_ids when num_target_layers <= 0. DFlash speculatively decodes by capturing target-model hidden states at selected intermediate layers, which requires knowing a positive number of target layers. This is a config plumbing error, not a runtime data error.

Source

Thrown at python/sglang/srt/speculative/dflash_utils.py:371

def build_target_layer_ids(num_target_layers: int, num_draft_layers: int) -> List[int]:
    """Select target layer indices used to build DFlash context features.

    Args:
        num_target_layers: Number of transformer layers in the runtime target model.
        num_draft_layers: Number of layers in the DFlash draft model.

    Returns:
        A list of 0-based target layer indices of length `num_draft_layers`.

    Notes:
        - DFlash uses hidden states after each selected target layer (HF-style).
        - SGLang captures "before layer i", so the model hook will typically add +1
          when mapping to capture points.
    """
    if num_target_layers <= 0:
        raise ValueError(
            f"num_target_layers must be positive, got {num_target_layers}."
        )
    if num_draft_layers <= 0:
        raise ValueError(f"num_draft_layers must be positive, got {num_draft_layers}.")

    if num_draft_layers == 1:
        return [num_target_layers // 2]

    start = 1
    end = num_target_layers - 3
    if end < start:
        raise ValueError(
            "DFlash layer selection requires num_target_layers >= 4. "
            f"Got num_target_layers={num_target_layers}."
        )

    span = end - start
    return [

View on GitHub (pinned to 0132848349)

Solutions

1. Pass the target model's actual num_hidden_layers (e.g. from config.text_config.num_hidden_layers).
2. If loading from a DFLASH draft config override, fix the num_hidden_layers/layer-count entry to a positive integer.
3. Inspect the HF config actually loaded (print config) to confirm where num_hidden_layers lives.

Example fix

# before
build_target_layer_ids(num_target_layers=0, num_draft_layers=2)
# after
build_target_layer_ids(num_target_layers=config.text_config.num_hidden_layers, num_draft_layers=2)

if not isinstance(num_target_layers, int) or num_target_layers <= 0:
    raise ValueError('num_target_layers must be a positive int')

def has_valid_layer_count(cfg) -> bool:
    n = getattr(getattr(cfg, 'text_config', cfg), 'num_hidden_layers', None)
    return isinstance(n, int) and n > 0

When it happens

Trigger: Calling build_target_layer_ids(num_target_layers=0 or negative, num_draft_layers=N), usually via resolve_target_layer_ids when the draft config lacks a valid hidden-layer count for the target model.

Common situations: Target model config has no num_hidden_layers (unusual architecture or partial config); a draft config JSON where the target layer count was typo'd as 0; text_config nesting not resolving so the layer count comes back as 0/None coerced.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• num_draft_layers must be positive, got {num_draft_layers}.
• DFlash layer selection requires num_target_layers >= 4. Got
• {field_name} must be {comparator}, got {parsed}.
• DFLASH mask_token must be a non-empty string, got {mask_toke
• Invalid fused KV rotary/head dim pair: rotary_dim={rotary_di

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/402b075a54fc5172. Report an issue: GitHub.