sgl-project/sglang · error · ValueError

Gemma4AssistantForCausalLM draft requires --speculative-algo

Error message

Gemma4AssistantForCausalLM draft requires --speculative-algorithm NEXTN or EAGLE; EAGLE3 is not supported for this draft architecture.

What it means

The Gemma4 assistant draft architectures (Gemma4AssistantForCausalLM / Gemma4UnifiedAssistantForCausalLM) only implement the NEXTN and EAGLE speculative heads; an EAGLE3 head is not defined for them, so SGLang rejects --speculative-algorithm EAGLE3 when the draft model is Gemma4.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:57

    kwargs: Optional[dict] = {},
) -> Optional[str]:
    """Resolve CLI speculative algorithm; NEXTN/EAGLE may become FROZEN_KV_MTP for Gemma4 assistant drafts."""

    is_gemma4_draft = False
    if speculative_draft_model_path:
        from sglang.srt.utils.hf_transformers_utils import get_config

        cfg = get_config(
            speculative_draft_model_path, trust_remote_code=trust_remote_code, **kwargs
        )
        draft_archs = getattr(cfg, "architectures", None) or []
        is_gemma4_draft = any(
            arch in ("Gemma4AssistantForCausalLM", "Gemma4UnifiedAssistantForCausalLM")
            for arch in draft_archs
        )

    if speculative_algorithm == "EAGLE3" and is_gemma4_draft:
        raise ValueError(
            "Gemma4AssistantForCausalLM draft requires "
            "--speculative-algorithm NEXTN or EAGLE; EAGLE3 is "
            "not supported for this draft architecture."
        )

    if speculative_algorithm == "NEXTN" or speculative_algorithm == "EAGLE":
        if is_gemma4_draft:
            logger.info(
                "Detected Gemma4AssistantForCausalLM draft; "
                f"promoting --speculative-algorithm {speculative_algorithm} to FROZEN_KV_MTP."
            )
            return "FROZEN_KV_MTP"
        return "EAGLE"

    return speculative_algorithm


def handle_speculative_decoding(server_args: ServerArgs) -> None:

View on GitHub (pinned to 0132848349)

Solutions

1. Use --speculative-algorithm NEXTN (typical for MTP-style drafts) or EAGLE
2. Switch to a model/draft with EAGLE3 support if EAGLE3 is required

Example fix

# before
--speculative-algorithm EAGLE3 --speculative-draft-model-path <gemma4-draft>
# after
--speculative-algorithm NEXTN --speculative-draft-model-path <gemma4-draft>

gemma_drafts = {"Gemma4AssistantForCausalLM", "Gemma4UnifiedAssistantForCausalLM"}
if speculative_algorithm == "EAGLE3" and set(draft_archs) & gemma_drafts:
    speculative_algorithm = "NEXTN"

def gemma4_supports(algo: str, draft_archs: list[str]) -> bool:
    if algo == "EAGLE3":
        return not set(draft_archs) & {"Gemma4AssistantForCausalLM", "Gemma4UnifiedAssistantForCausalLM"}
    return True

When it happens

Trigger: --speculative-algorithm EAGLE3 with a draft/model config whose architectures include Gemma4AssistantForCausalLM or Gemma4UnifiedAssistantForCausalLM.

Common situations: Swapping speculative algorithm to EAGLE3 for quality on a Gemma4 MTP-style deployment; reusing an EAGLE3 config from a Qwen/DeepSeek setup.

Related errors

• Kimi-K3 DCP + DSPARK currently requires SGLANG_RAGGED_VERIFY
• --disaggregation-decode-enable-radix-cache is incompatible w
• --speculative-draft-window-size must be positive, got {}.
• DFLASH speculative decoding only supports CUDA and NPU devic
• trtllm_mla cannot serve decode context parallelism with spec

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1a9be5600ca02935. Report an issue: GitHub.