sgl-project/sglang · error · ValueError

EAGLE3 currently only supports 1 layer

Error message

EAGLE3 currently only supports 1 layer

What it means

The EAGLE3 draft model for Kimi K2.5 is hardcoded to exactly one decoder layer (the single midlayer built at layer_id=0). The EAGLE3 paper's feature-extraction draft uses one layer, so multi-layer draft configs are rejected up front with this ValueError rather than silently mis-loading weights.

Source

Thrown at python/sglang/srt/models/kimi_k25_eagle3.py:243

        )

        # Per-aux RMSNorm before fc; enabled via `fc_norm` or legacy
        # `use_aux_norm` flag. Matches the eagle3.1 layout.
        use_fc_norm = getattr(config, "fc_norm", None) or getattr(
            config, "use_aux_norm", False
        )
        if use_fc_norm:
            self.fc_norm = nn.ModuleList(
                [
                    RMSNorm(target_hidden_size, eps=config.rms_norm_eps)
                    for _ in range(self.num_aux_hidden_states)
                ]
            )
        else:
            self.fc_norm = None

        if config.num_hidden_layers != 1:
            raise ValueError("EAGLE3 currently only supports 1 layer")
        self.midlayer = Eagle3MLADecoderLayer(
            config,
            layer_id=0,
            quant_config=quant_config,
            prefix=prefix,
        )

        self.norm = RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
        # Draft decode captures pre-norm hidden by default; eagle3.1 opts for
        # post-norm via `norm_output: true`.
        self.norm_output = getattr(config, "norm_output", False)

    def forward(
        self,
        input_ids: torch.Tensor,
        positions: torch.Tensor,
        forward_batch: ForwardBatch,
        input_embeds: torch.Tensor = None,

View on GitHub (pinned to 0132848349)

Solutions

1. Set num_hidden_layers to 1 in the EAGLE3 draft model's config.json.
2. Use an official/validated single-layer EAGLE3 draft checkpoint for Kimi K2.5.
3. If multi-layer drafting is needed, use the standard EAGLE (non-EAGLE3) draft path which supports multiple layers.

Example fix

// before (draft config.json)
{"num_hidden_layers": 4, ...}

// after
{"num_hidden_layers": 1, ...}

assert json.load(open(draft_config_path))["num_hidden_layers"] == 1, "EAGLE3 draft must have exactly 1 layer"

When it happens

Trigger: Loading an EAGLE3 draft model whose config.json has num_hidden_layers != 1 (e.g. 2 or more) — the model builds a single Eagle3MLADecoderLayer and there is no code path for additional layers.

Common situations: Using a community EAGLE3 draft checkpoint with extra layers; editing a draft config and bumping num_hidden_layers; accidentally passing a full model (many layers) as --speculative-draft-model-path.

Related errors

• Eagle3 MLA layer requires q_lora_rank in the draft config
• Invalid fused KV rotary/head dim pair: rotary_dim={rotary_di
• num_kv_heads mismatch across layers for fused KV path: expec
• Gemma4AssistantForCausalLM draft requires --speculative-algo
• language_model does not support get_embed_and_head().

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/86f44c5fccbab541. Report an issue: GitHub.