{"record":{"id":"1ef9b77d5d495def","repo":"sgl-project/sglang","slug":"eagle3-mla-layer-requires-q-lora-rank-in-the-draft","errorCode":null,"errorMessage":"Eagle3 MLA layer requires q_lora_rank in the draft config","messagePattern":"Eagle3 MLA layer requires q_lora_rank in the draft config","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/kimi_k25_eagle3.py","lineNumber":114,"sourceCode":"            v_head_dim=config.v_head_dim,\n            q_lora_rank=config.q_lora_rank,\n            kv_lora_rank=config.kv_lora_rank,\n            rope_theta=rope_theta,\n            rope_scaling=rope_scaling,\n            max_position_embeddings=max_position_embeddings,\n            quant_config=quant_config,\n            layer_id=layer_id,\n            reduce_results=True,\n            prefix=add_prefix(\"self_attn\", prefix),\n        )\n\n        # EAGLE3 doubles MLA's QKV-down input by concatenating\n        # input_layernorm(embed) and hidden_norm(target_hidden) along the\n        # feature dim. Replace the projection that DeepseekV2AttentionMLA\n        # built for a single-hidden input.\n        attn = self.self_attn\n        if attn.q_lora_rank is None:\n            raise ValueError(\n                \"Eagle3 MLA layer requires q_lora_rank in the draft config\"\n            )\n        attn.fused_qkv_a_proj_with_mqa = ReplicatedLinear(\n            2 * config.hidden_size,\n            attn.q_lora_rank + attn.kv_lora_rank + attn.qk_rope_head_dim,\n            bias=False,\n            quant_config=quant_config,\n            prefix=add_prefix(\"self_attn.fused_qkv_a_proj_with_mqa\", prefix),\n        )\n        # Recompute fused-proj-dependent flags so they reflect the new input dim.\n        attn.has_fused_proj = True\n        attn._use_min_latency_fused_a_gemm = False\n        quant_method = getattr(attn.fused_qkv_a_proj_with_mqa, \"quant_method\", None)\n        attn.is_packed_weight = (\n            quant_method is not None\n            and hasattr(quant_method, \"quant_config\")\n            and quant_method.quant_config is not None\n            and quant_method.quant_config.get_name()","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/kimi_k25_eagle3.py#L96-L132","documentation":"The EAGLE3 draft layer for Kimi K2.5 rebuilds the MLA fused QKV-down projection to accept 2*hidden_size inputs (concatenated embed + target hidden states). This requires a q_lora_rank in the draft config because the projection output size is derived from it. Without it, DeepseekV2AttentionMLA built an MQA-style projection and this ValueError fires at construction.","triggerScenarios":"Instantiating Eagle3MLADecoderLayer (Eagle3MLA draft model __init__) with a draft config where config.q_lora_rank is None — i.e. a pure-MLA (MQA) draft model without Q low-rank compression, such as a K2.5 MTP head used as the EAGLE3 draft.","commonSituations":"Pointing --speculative-draft-model-path at a Kimi MTP draft checkpoint whose config lacks q_lora_rank; hand-written EAGLE3 draft configs missing the q_lora_rank field; mixing a DeepSeek-V2-style MLA config with an EAGLE3 wrapper that assumes Q-LoRA is present.","solutions":["Set q_lora_rank (a positive int, e.g. 1536) in the draft model's config.json when using the EAGLE3 MLA draft.","Use a draft checkpoint that was built for EAGLE3 (has q_lora_rank) rather than the plain MTP head.","Verify the draft config loads with q_lora_rank != None before launching: json.load(config)['q_lora_rank'] is not None."],"exampleFix":"// before (draft config.json)\n{\"hidden_size\": 7168, \"kv_lora_rank\": 512, ...}\n\n// after\n{\"hidden_size\": 7168, \"q_lora_rank\": 1536, \"kv_lora_rank\": 512, ...}","handlingStrategy":"validation","validationCode":"import json\ncfg = json.load(open(draft_config_path))\nif cfg.get(\"q_lora_rank\") is None:\n    raise ValueError(\"EAGLE3 MLA draft requires q_lora_rank in draft config\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate draft config fields (q_lora_rank, num_hidden_layers) before launching the server.","Use draft checkpoints specifically built for EAGLE3."],"tags":["eagle3","speculative-decoding","mla","config-validation","kimi"],"backgroundTag":"model-config-missing-field","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}