sgl-project/sglang · error · AttributeError

language_model does not support set_embed_and_head().

Error message

language_model does not support set_embed_and_head().

What it means

KimiK25ForCausalLM.set_embed_and_head() restores embedding and LM-head weights on the wrapped language_model, the counterpart to get_embed_and_head used by speculative decoding (e.g. temporarily tying/untying weights). It raises AttributeError when the delegate method is absent or language_model is None, so the weight-swap protocol cannot proceed.

Source

Thrown at python/sglang/srt/models/kimi_k25.py:979

        return self.language_model.lm_head

    def get_embed_and_head(self) -> Tuple[torch.Tensor, torch.Tensor]:
        """Get embedding and LM head weights for speculative decoding."""
        if self.language_model is None or not hasattr(
            self.language_model, "get_embed_and_head"
        ):
            raise AttributeError(
                "language_model does not support get_embed_and_head()."
            )

        return self.language_model.get_embed_and_head()

    def set_embed_and_head(self, embed: torch.Tensor, head: torch.Tensor) -> None:
        """Set embedding and LM head weights for speculative decoding."""
        if self.language_model is None or not hasattr(
            self.language_model, "set_embed_and_head"
        ):
            raise AttributeError(
                "language_model does not support set_embed_and_head()."
            )

        self.language_model.set_embed_and_head(embed, head)


EntryClass = [KimiK25ForConditionalGeneration]

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure both get_embed_and_head and set_embed_and_head exist on the inner language_model class (implement them in pairs).
2. Confirm language_model initialization succeeded before starting spec decoding.
3. Disable speculative decoding if the inner model only partially implements the weight-access protocol.

Example fix

// before
model.set_embed_and_head(embed, head)  # AttributeError

// after
lm = model.language_model
if lm is not None and hasattr(lm, "set_embed_and_head"):
    lm.set_embed_and_head(embed, head)

lm = model.language_model
assert lm is not None and hasattr(lm, "set_embed_and_head"), "spec weight set unsupported"

def supports_embed_head_rw(model) -> bool:
    lm = getattr(model, "language_model", None)
    return lm is not None and all(hasattr(lm, m) for m in ("get_embed_and_head", "set_embed_and_head"))

When it happens

Trigger: The speculative worker calls set_embed_and_head(embed, head) after get_embed_and_head on a wrapper whose language_model is None or whose class lacks set_embed_and_head — same conditions as error 5420 but on the write path.

Common situations: Same as get_embed_and_head: enabling EAGLE/MTP speculative decoding against a Kimi K2.5 variant without the spec-decode hooks; inconsistent model implementations that define one of get/set but not both.

Related errors

• language_model does not support get_embed_and_head().
• Eagle3 MLA layer requires q_lora_rank in the draft config
• EAGLE3 currently only supports 1 layer
• Unsupported activation: {hidden_act}
• DSPARK aux hidden capture requires PP=1.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1fa5da28886c7974. Report an issue: GitHub.