sgl-project/sglang · error · AttributeError

DSPARK layer capture is not available in encoder-only mode

Error message

DSPARK layer capture is not available in encoder-only mode

What it means

set_dspark_layers_to_capture() raises AttributeError when called on a Kimi-K3 model that has no language model (encoder-only mode), since DSPARK layer capture applies to language-model layers only. It is a pass-through to the language model's capture API and cannot work without one.

Source

Thrown at python/sglang/srt/models/kimi_k3.py:3324

        if self.vision_tower.precompile_attention_backend():
            logger.info("Precompiled Kimi-K3 vision FA4 kernel")

    def get_input_embeddings(self):
        if self.language_model is None:
            raise AttributeError(
                "get_input_embeddings() is not available in encoder-only mode"
            )
        return self.language_model.model.embed_tokens

    @property
    def lm_head(self):
        if self.language_model is None:
            raise AttributeError("lm_head is not available in encoder-only mode")
        return self.language_model.lm_head

    def set_dspark_layers_to_capture(self, layer_ids: list[int]) -> None:
        if self.language_model is None:
            raise AttributeError(
                "DSPARK layer capture is not available in encoder-only mode"
            )
        self.language_model.set_dspark_layers_to_capture(layer_ids)

    def preprocess_mm_for_encoder(
        self,
        mm_data,
        modality,
        config,
        *,
        image_processor=None,
        use_gpu_preprocessing=False,
    ):
        """Prepare per-image raw inputs for owner-side EPD preprocessing."""
        if modality != Modality.IMAGE:
            raise ValueError("Kimi-K3 encoder mode supports image input only")
        if image_processor is None:
            raise ValueError("Kimi-K3 encoder preprocessing needs an image processor")

View on GitHub (pinned to 0132848349)

Solutions

Only call set_dspark_layers_to_capture when model.language_model is not None
Configure DSPARK capture on the decoder/prefill server, not the encoder-only instance
Gate the call site behind an encoder-only check

Example fix

// before
model.set_dspark_layers_to_capture([8, 16, 24])

// after
if model.language_model is not None:
    model.set_dspark_layers_to_capture([8, 16, 24])

Defensive patterns

Strategy: validation

Validate before calling

if model.language_model is not None:
    model.set_dspark_layers_to_capture(layer_ids)

Type guard

def supports_dspark_capture(m) -> bool:
    return getattr(m, "language_model", None) is not None

Try / catch

try:
    model.set_dspark_layers_to_capture(ids)
except AttributeError:
    logger.warning("DSPARK capture skipped: encoder-only model")

Prevention

Configure speculative/DSPARK capture only on decoder-side processes
Centralize capability checks instead of probing methods with try/except

When it happens

Trigger: Calling model.set_dspark_layers_to_capture(layer_ids) on an encoder-only Kimi-K3 instance, typically from speculative-decoding or layer-capture setup that runs on every model.

Common situations: Enabling DSPARK/speculative capture in a disaggregated EPD setup where the encoder process loads only the vision tower; generic harness code that calls capture setup unconditionally.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/25ba27ae51800152. Report an issue: GitHub.