{"record":{"id":"1a9be5600ca02935","repo":"sgl-project/sglang","slug":"gemma4assistantforcausallm-draft-requires-specul","errorCode":null,"errorMessage":"Gemma4AssistantForCausalLM draft requires --speculative-algorithm NEXTN or EAGLE; EAGLE3 is not supported for this draft architecture.","messagePattern":"Gemma4AssistantForCausalLM draft requires --speculative-algorithm NEXTN or EAGLE; EAGLE3 is not supported for this draft architecture\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/arg_groups/speculative_hook.py","lineNumber":57,"sourceCode":"    kwargs: Optional[dict] = {},\n) -> Optional[str]:\n    \"\"\"Resolve CLI speculative algorithm; NEXTN/EAGLE may become FROZEN_KV_MTP for Gemma4 assistant drafts.\"\"\"\n\n    is_gemma4_draft = False\n    if speculative_draft_model_path:\n        from sglang.srt.utils.hf_transformers_utils import get_config\n\n        cfg = get_config(\n            speculative_draft_model_path, trust_remote_code=trust_remote_code, **kwargs\n        )\n        draft_archs = getattr(cfg, \"architectures\", None) or []\n        is_gemma4_draft = any(\n            arch in (\"Gemma4AssistantForCausalLM\", \"Gemma4UnifiedAssistantForCausalLM\")\n            for arch in draft_archs\n        )\n\n    if speculative_algorithm == \"EAGLE3\" and is_gemma4_draft:\n        raise ValueError(\n            \"Gemma4AssistantForCausalLM draft requires \"\n            \"--speculative-algorithm NEXTN or EAGLE; EAGLE3 is \"\n            \"not supported for this draft architecture.\"\n        )\n\n    if speculative_algorithm == \"NEXTN\" or speculative_algorithm == \"EAGLE\":\n        if is_gemma4_draft:\n            logger.info(\n                \"Detected Gemma4AssistantForCausalLM draft; \"\n                f\"promoting --speculative-algorithm {speculative_algorithm} to FROZEN_KV_MTP.\"\n            )\n            return \"FROZEN_KV_MTP\"\n        return \"EAGLE\"\n\n    return speculative_algorithm\n\n\ndef handle_speculative_decoding(server_args: ServerArgs) -> None:","sourceCodeStart":39,"sourceCodeEnd":75,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/arg_groups/speculative_hook.py#L39-L75","documentation":"The Gemma4 assistant draft architectures (Gemma4AssistantForCausalLM / Gemma4UnifiedAssistantForCausalLM) only implement the NEXTN and EAGLE speculative heads; an EAGLE3 head is not defined for them, so SGLang rejects --speculative-algorithm EAGLE3 when the draft model is Gemma4.","triggerScenarios":"--speculative-algorithm EAGLE3 with a draft/model config whose architectures include Gemma4AssistantForCausalLM or Gemma4UnifiedAssistantForCausalLM.","commonSituations":"Swapping speculative algorithm to EAGLE3 for quality on a Gemma4 MTP-style deployment; reusing an EAGLE3 config from a Qwen/DeepSeek setup.","solutions":["Use --speculative-algorithm NEXTN (typical for MTP-style drafts) or EAGLE","Switch to a model/draft with EAGLE3 support if EAGLE3 is required"],"exampleFix":"# before\n--speculative-algorithm EAGLE3 --speculative-draft-model-path <gemma4-draft>\n# after\n--speculative-algorithm NEXTN --speculative-draft-model-path <gemma4-draft>","handlingStrategy":"validation","validationCode":"gemma_drafts = {\"Gemma4AssistantForCausalLM\", \"Gemma4UnifiedAssistantForCausalLM\"}\nif speculative_algorithm == \"EAGLE3\" and set(draft_archs) & gemma_drafts:\n    speculative_algorithm = \"NEXTN\"","typeGuard":"def gemma4_supports(algo: str, draft_archs: list[str]) -> bool:\n    if algo == \"EAGLE3\":\n        return not set(draft_archs) & {\"Gemma4AssistantForCausalLM\", \"Gemma4UnifiedAssistantForCausalLM\"}\n    return True","tryCatchPattern":null,"preventionTips":["Key speculative-algorithm choice off the draft model's architectures"],"tags":["sglang","speculative-decoding","eagle3","gemma4","model-incompatibility"],"backgroundTag":"model-architecture-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}