sgl-project/sglang · error · RuntimeError

MLX model has no supported attention layers

Error message

MLX model has no supported attention layers

What it means

`MlxModelCacheLayout.first_attention_layer_index` returns the index of the first attention layer to seed things like the KV pool. If the layout has no attention layers at all (all attrs were None during discovery), there is no meaningful answer and it raises RuntimeError.

Source

Thrown at python/sglang/srt/hardware_backend/mlx/kv_cache/layout.py:108

    def num_attention_layers(self) -> int:
        return len(self.attention_layer_indices)

    @property
    def num_full_attention_layers(self) -> int:
        return len(self.full_attention_layer_indices)

    @property
    def has_sliding_window_layers(self) -> bool:
        return bool(self.swa_attention_layer_indices)

    @property
    def has_auxiliary_state(self) -> bool:
        return bool(self.auxiliary_layer_indices)

    @property
    def first_attention_layer_index(self) -> int:
        if not self.attention_layer_indices:
            raise RuntimeError("MLX model has no supported attention layers")
        return self.attention_layer_indices[0]

    def window_size(self, layer_idx: int) -> int | None:
        """Sliding window of *layer_idx*, or None for full attention."""
        return self.layer_window_sizes.get(layer_idx)

    def attention_pool_index(self, layer_idx: int) -> int:
        try:
            return self.attention_pool_index_by_layer[layer_idx]
        except KeyError as exc:
            raise KeyError(f"Layer {layer_idx} is not an attention layer") from exc

    def full_kv_pool_index(self, layer_idx: int) -> int:
        try:
            return self.full_kv_pool_index_by_layer[layer_idx]
        except KeyError as exc:
            raise KeyError(f"Layer {layer_idx} is not a full-attention layer") from exc

View on GitHub (pinned to 0132848349)

Solutions

1. Check `layout.attention_layer_indices` (or has_attention) before accessing the property.
2. Fix attention discovery for the model so attention_attrs contains real attribute names.
3. For Mamba-only models, use the auxiliary-state path instead of the KV-cache layout path.

Example fix

# before
idx = layout.first_attention_layer_index  # RuntimeError

# after
idx = (
    layout.first_attention_layer_index
    if layout.attention_layer_indices
    else None
)

if not layout.attention_layer_indices:
    raise ValueError("no attention layers discovered; check discovery attrs")
idx = layout.first_attention_layer_index

def has_attention(layout) -> bool:
    return bool(layout.attention_layer_indices)

When it happens

Trigger: Accessing first_attention_layer_index on a layout built with all-None attention_attrs — e.g. discovery failed to recognize any attention module (unsupported architecture) or a Mamba-only model was passed.

Common situations: Running a pure linear-attention/Mamba model through the MLX KV-cache layout path; a rename in mlx_lm so no attention attrs match; wrapping a model whose attention layers live in a non-standard container.

Related errors

• Layer count and attention attribute count differ: {len(layer
• AttentionOffsetCache should not store data
• WindowedAttentionKVCache holds only the trailing window and
• BatchedDecodeContext requires full_kv_pool_index_by_layer wh
• Cannot determine attention scale for {type(inner).__name__}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0cfa6dd7b9d10bcc. Report an issue: GitHub.