sgl-project/sglang · error · AttributeError

get_input_embeddings() is not available in encoder-only mode

Error message

get_input_embeddings() is not available in encoder-only mode

What it means

Kimi-K3 model wrapper raises AttributeError from get_input_embeddings() when the model was constructed in encoder-only mode (language_model is None). Encoder-only instances have no language model, hence no token embedding table to return. Any generic model-loading or weight-mapping code that unconditionally calls get_input_embeddings() will trip this.

Source

Thrown at python/sglang/srt/models/kimi_k3.py:3311

            return
        super().__setattr__(name, value)

    def post_load_weights(self):
        # Delegate so DummyModelLoader's post-load hook reaches the LM tower.
        if self.language_model is not None:
            self.language_model.post_load_weights()

    def precompile_kernels_after_loading(self) -> None:
        if self.config.language_only:
            return
        if self.vision_tower.precompile_fused_rope():
            logger.info("Precompiled dynamic-token fused K3 vision RoPE kernel")
        if self.vision_tower.precompile_attention_backend():
            logger.info("Precompiled Kimi-K3 vision FA4 kernel")

    def get_input_embeddings(self):
        if self.language_model is None:
            raise AttributeError(
                "get_input_embeddings() is not available in encoder-only mode"
            )
        return self.language_model.model.embed_tokens

    @property
    def lm_head(self):
        if self.language_model is None:
            raise AttributeError("lm_head is not available in encoder-only mode")
        return self.language_model.lm_head

    def set_dspark_layers_to_capture(self, layer_ids: list[int]) -> None:
        if self.language_model is None:
            raise AttributeError(
                "DSPARK layer capture is not available in encoder-only mode"
            )
        self.language_model.set_dspark_layers_to_capture(layer_ids)

    def preprocess_mm_for_encoder(

View on GitHub (pinned to 0132848349)

Solutions

1. Check hasattr(model, 'language_model') / model.language_model is not None before calling get_input_embeddings()
2. Skip embedding probing for encoder-only model configs in generic tooling
3. If you expected a full VLM, verify the load config did not disable the language model

Example fix

// before
embed = model.get_input_embeddings()

// after
if model.language_model is not None:
    embed = model.get_input_embeddings()
else:
    embed = None  # encoder-only mode

if model.language_model is None:
    skip_embedding_related_setup(model)

def has_language_model(m) -> bool:
    return getattr(m, "language_model", None) is not None

try:
    emb = model.get_input_embeddings()
except AttributeError:
    emb = None  # encoder-only instance

When it happens

Trigger: Instantiating the Kimi-K3 model in encoder-only mode (e.g. EPD encoder server without the language model) and then calling model.get_input_embeddings(), directly or via generic weight-loading/tied-embedding logic that assumes a decoder exists.

Common situations: Running the encoder side of encoder-prefill-decoder (EPD) disaggregation; reusing generic model utilities that probe embed_tokens for vocab size or tied weights on a vision-only instance.

Related errors

• lm_head is not available in encoder-only mode
• DSPARK layer capture is not available in encoder-only mode
• module {__name__!r} has no attribute {name!r}
• attn_res: nvb must be in [1, {_MAX_BANK_ROWS}], got {nvb}
• Unsupported text encoder output: expected `hidden_states`.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/22d7f75a3d41ad96. Report an issue: GitHub.