sgl-project/sglang · error · ValueError

DFLASH requires draft num_hidden_layers in config. Got confi

Error message

DFLASH requires draft num_hidden_layers in config. Got config without num_hidden_layers.

What it means

Raised by DFlashDraftConfig.require_num_layers when num_hidden_layers is None. Building DFlash target-layer ids (and resolving aux hidden-state capture points) needs the draft model's layer count; its absence means the draft config never supplied num_hidden_layers. It is an explicit, fail-fast guard against silently guessing a layer layout.

Source

Thrown at python/sglang/srt/speculative/dflash_utils.py:543

@dataclass(frozen=True)
class DFlashDraftConfig:
    num_hidden_layers: Optional[int]
    num_target_layers: Optional[int]
    block_size: Optional[int]
    conv_kernel_size: int
    conv_group_size: int
    selector_rank: int
    selector_top_k: int
    output_multiplier: float
    final_logit_softcapping: Optional[float]
    target_layer_ids: Optional[List[int]]
    mask_token: str
    mask_token_id: Optional[int]

    def require_num_layers(self) -> int:
        if self.num_hidden_layers is None:
            raise ValueError(
                "DFLASH requires draft num_hidden_layers in config. "
                "Got config without num_hidden_layers."
            )
        return int(self.num_hidden_layers)

    def resolve_block_size(self, *, default: Optional[int] = None) -> Optional[int]:
        return self.block_size if self.block_size is not None else default

    def resolve_target_layer_ids(
        self,
        *,
        target_num_layers: int,
        draft_num_layers: Optional[int] = None,
    ) -> List[int]:
        target_num_layers = int(target_num_layers)
        if target_num_layers <= 0:
            raise ValueError(
                f"target_num_layers must be positive, got {target_num_layers}."

View on GitHub (pinned to 0132848349)

Solutions

1. Add num_hidden_layers (the draft model's layer count) to the DFLASH draft config.
2. If the count exists under another key in the HF config, fix the config-reading code/path so it is picked up when building DFlashDraftConfig.
3. Alternatively pass num_hidden_layers explicitly wherever the config object is constructed.

Example fix

# before
# draft_config.json: {"draft_model": "...", "mask_token": "[MASK]"}  (no num_hidden_layers)
# after
# draft_config.json: {"draft_model": "...", "mask_token": "[MASK]", "num_hidden_layers": 4}

if draft_config.num_hidden_layers is None:
    raise ValueError('DFLASH draft config must include num_hidden_layers')

def has_num_hidden_layers(cfg) -> bool:
    return getattr(cfg, 'num_hidden_layers', None) is not None

When it happens

Trigger: Constructing DFlashDraftConfig without num_hidden_layers (e.g. parsed from a draft config JSON missing the field) and then calling require_num_layers() directly or via resolve_target_layer_ids / _resolve_dflash_aux_hidden_state.

Common situations: DFLASH draft config in the HF repo lacks num_hidden_layers; a locally crafted draft config omitted the field; a nested text_config where the layer count lives at a different key and the parser did not find it.

Related errors

• DFLASH speculative decoding only supports CUDA and NPU devic
• Currently DFLASH speculative decoding does not support dp at
• Currently DFLASH speculative decoding only supports pp_size
• DFLASH speculative decoding requires setting --speculative-d
• DFLASH requires --speculative-dflash-block-size to be positi

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6cb40f2abae50ec3. Report an issue: GitHub.