{"record":{"id":"3cfbcc7249ab7f07","repo":"sgl-project/sglang","slug":"dflash-layer-selection-requires-num-target-layers","errorCode":null,"errorMessage":"DFlash layer selection requires num_target_layers >= 4. Got num_target_layers={num_target_layers}.","messagePattern":"DFlash layer selection requires num_target_layers >= 4\\. Got num_target_layers=(.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/speculative/dflash_utils.py","lineNumber":383,"sourceCode":"    Notes:\n        - DFlash uses hidden states after each selected target layer (HF-style).\n        - SGLang captures \"before layer i\", so the model hook will typically add +1\n          when mapping to capture points.\n    \"\"\"\n    if num_target_layers <= 0:\n        raise ValueError(\n            f\"num_target_layers must be positive, got {num_target_layers}.\"\n        )\n    if num_draft_layers <= 0:\n        raise ValueError(f\"num_draft_layers must be positive, got {num_draft_layers}.\")\n\n    if num_draft_layers == 1:\n        return [num_target_layers // 2]\n\n    start = 1\n    end = num_target_layers - 3\n    if end < start:\n        raise ValueError(\n            \"DFlash layer selection requires num_target_layers >= 4. \"\n            f\"Got num_target_layers={num_target_layers}.\"\n        )\n\n    span = end - start\n    return [\n        int(round(start + (i * span) / (num_draft_layers - 1)))\n        for i in range(num_draft_layers)\n    ]\n\n\ndef get_dflash_layer_types(config: Any) -> Optional[Sequence[str]]:\n    text_config = _get_text_config(config)\n    layer_types = _cfg_get(text_config, \"layer_types\", _cfg_get(config, \"layer_types\"))\n    if layer_types is None:\n        return None\n    if isinstance(layer_types, str) or not isinstance(layer_types, Sequence):\n        raise ValueError(","sourceCodeStart":365,"sourceCodeEnd":401,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/speculative/dflash_utils.py#L365-L401","documentation":"Raised by build_target_layer_ids when num_target_layers < 4 and more than one draft layer is requested. The multi-draft-layer placement picks layers in the range [1, num_target_layers-3], which is empty for tiny models. Single-draft-layer configs bypass this (they use the midpoint), so only multi-layer drafts on very small models hit it.","triggerScenarios":"Calling build_target_layer_ids with num_target_layers in {1,2,3} (end = num_target_layers-3 < start=1) and num_draft_layers >= 2, e.g. a 2-layer toy target model with a 2-layer draft.","commonSituations":"Running DFlash speculative decoding on tiny test/debug models (1-3 layers); unit tests with miniature configs requesting multiple draft layers; misreading num_hidden_layers (e.g. passing layer index instead of count).","solutions":["Use num_draft_layers=1 for tiny target models (it falls back to the midpoint layer).","Use a target model with at least 4 hidden layers for multi-layer drafts.","Double-check you passed num_hidden_layers (a count like 32), not a layer index or per-block count."],"exampleFix":"# before\nbuild_target_layer_ids(num_target_layers=3, num_draft_layers=2)\n# after\nbuild_target_layer_ids(num_target_layers=3, num_draft_layers=1)","handlingStrategy":"fallback","validationCode":"if num_draft_layers > 1 and num_target_layers < 4:\n    num_draft_layers = 1  # fall back to single midpoint layer\n# or reject early:\nif num_draft_layers > 1 and num_target_layers < 4:\n    raise ValueError('multi-layer DFlash draft needs >= 4 target layers')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Gate multi-draft-layer configs on num_target_layers >= 4 in config validation.","Avoid tiny (<4 layer) target models for DFlash drafts with multiple draft layers."],"tags":["speculative-decoding","dflash","layer-selection","config-validation"],"backgroundTag":"model-config-incompatible","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}