sgl-project/sglang · error · AttributeError

language_model does not support get_embed_and_head().

Error message

language_model does not support get_embed_and_head().

What it means

KimiK25ForCausalLM.get_embed_and_head() delegates to the wrapped language_model to fetch embedding and LM-head weights for speculative decoding. If language_model is None or lacks its own get_embed_and_head method, this AttributeError is raised. It signals that the model wrapper is not compatible with the speculative-decoding weight-access API.

Source

Thrown at python/sglang/srt/models/kimi_k25.py:968

            raise AttributeError(
                "language_model does not support get_input_embeddings()."
            )

        return self.language_model.get_input_embeddings()

    @property
    def lm_head(self):
        if not hasattr(self.language_model, "lm_head"):
            raise AttributeError("language_model does not expose lm_head.")

        return self.language_model.lm_head

    def get_embed_and_head(self) -> Tuple[torch.Tensor, torch.Tensor]:
        """Get embedding and LM head weights for speculative decoding."""
        if self.language_model is None or not hasattr(
            self.language_model, "get_embed_and_head"
        ):
            raise AttributeError(
                "language_model does not support get_embed_and_head()."
            )

        return self.language_model.get_embed_and_head()

    def set_embed_and_head(self, embed: torch.Tensor, head: torch.Tensor) -> None:
        """Set embedding and LM head weights for speculative decoding."""
        if self.language_model is None or not hasattr(
            self.language_model, "set_embed_and_head"
        ):
            raise AttributeError(
                "language_model does not support set_embed_and_head()."
            )

        self.language_model.set_embed_and_head(embed, head)


EntryClass = [KimiK25ForConditionalGeneration]

View on GitHub (pinned to 0132848349)

Solutions

1. Verify self.language_model is constructed (not None) before enabling speculative decoding.
2. Check the inner language model class implements get_embed_and_head (see Qwen3/deepseek wrappers for reference) and add the delegation method if missing.
3. If the inner model is not spec-decode compatible, disable speculative decoding for this model.
4. Upgrade sglang to a version where the Kimi K2.5 language model implements the spec-decoding weight hooks.

Example fix

// before
embed, head = model.get_embed_and_head()  # AttributeError

// after
lm = model.language_model
if lm is None or not hasattr(lm, "get_embed_and_head"):
    raise RuntimeError("speculative decoding unsupported for this language model")
embed, head = lm.get_embed_and_head()

def supports_spec_weights(model) -> bool:
    lm = getattr(model, "language_model", None)
    return lm is not None and hasattr(lm, "get_embed_and_head")

def has_get_embed_and_head(model) -> bool:
    lm = getattr(model, "language_model", None)
    return lm is not None and callable(getattr(lm, "get_embed_and_head", None))

When it happens

Trigger: Calling model.get_embed_and_head() on a Kimi K2.5 wrapper whose self.language_model is None (e.g. draft-model wiring skipped) or whose language_model class (e.g. a MTP/module variant) does not define get_embed_and_head. Typically invoked by the EAGLE/speculative spec worker during draft setup.

Common situations: Enabling speculative decoding (EAGLE/MTP) with a target model whose inner language model implementation doesn't implement the spec-decoding hooks; partial initialization where the language model failed to construct; version skew after refactoring model classes.

Related errors

• language_model does not support set_embed_and_head().
• Eagle3 MLA layer requires q_lora_rank in the draft config
• EAGLE3 currently only supports 1 layer
• Unsupported activation: {hidden_act}
• DSPARK aux hidden capture requires PP=1.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/976f2ec0c967fe9d. Report an issue: GitHub.