sgl-project/sglang · critical · TypeError

Layer {} is not backed by an MLA KV pool

Error message

Layer {} is not backed by an MLA KV pool

What it means

SWA memory pool's set_mla_kv_buffer resolved the layer's backing pool and it is not an MLATokenToKVPool, so the MLA buffer API cannot be used for that layer.

Source

Thrown at python/sglang/srt/mem_cache/swa_memory_pool.py:285

                v_scale,
                layer_id_override=layer_id_pool,
            )

    def set_mla_kv_buffer(
        self,
        layer: RadixAttention,
        loc_info,
        cache_k_nope: torch.Tensor,
        cache_k_rope: torch.Tensor,
    ):
        loc, swa_loc, _ = unwrap_write_loc(loc_info)
        layer_id_pool, is_swa_layer = self.layers_mapping[layer.layer_id]
        pool = self.swa_kv_pool if is_swa_layer else self.full_kv_pool
        if is_swa_layer:
            assert swa_loc is not None
            loc = swa_loc
        if not isinstance(pool, MLATokenToKVPool):
            raise TypeError(f"Layer {layer.layer_id} is not backed by an MLA KV pool")
        pool.set_mla_kv_buffer(
            None,
            loc,
            cache_k_nope,
            cache_k_rope,
            layer_id_override=layer_id_pool,
        )

    def get_index_k_with_scale_buffer(self, layer_id: int) -> torch.Tensor:
        layer_id_pool, is_swa_layer = self.layers_mapping[layer_id]
        assert not is_swa_layer
        return self.full_kv_pool.get_index_k_with_scale_buffer(layer_id_pool)

    def get_index_k_continuous(self, layer_id: int, *args, **kwargs):
        layer_id_pool, is_swa_layer = self.layers_mapping[layer_id]
        assert not is_swa_layer
        return self.full_kv_pool.get_index_k_continuous(layer_id_pool, *args, **kwargs)

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure the model is actually MLA and the pools passed to the SWA pool are MLATokenToKVPool instances
2. If using custom pools, implement the MLATokenToKVPool interface (set_mla_kv_buffer)
3. Check layers_mapping/layer_id routing so MLA layers map to MLA-backed pools

Example fix

# before
pool = SWAKVPool(full_kv_pool=MHATokenToKVPool(...), ...)
pool.set_mla_kv_buffer(layer, ...)
# after
from sglang.srt.mem_cache.memory_pool import MLATokenToKVPool
pool = SWAKVPool(full_kv_pool=MLATokenToKVPool(...), ...)
pool.set_mla_kv_buffer(layer, ...)

from sglang.srt.mem_cache.memory_pool import MLATokenToKVPool
assert isinstance(pool.full_kv_pool, MLATokenToKVPool) and isinstance(pool.swa_kv_pool, MLATokenToKVPool)

def is_mla_pool(p) -> bool:
    from sglang.srt.mem_cache.memory_pool import MLATokenToKVPool
    return isinstance(p, MLATokenToKVPool)

When it happens

Trigger: Constructing SWAKVPool with a full/swa pool that is a MHATokenToKVPool (or other non-MLA pool) and then calling set_mla_kv_buffer for a layer mapped to it — typical when an MLA model config is mismatched with non-MLA pool classes.

Common situations: Mixing MLA model support with non-MLA allocator classes, custom pool implementations missing the MLA interface, or wrong layers_mapping wiring.

Related errors

• Unsupported type {type(data)}
• All tensors must have the same data type
• sparse_mla_q8kv8_prefill_fwd requires h_kv=1, got {h_kv}
• sparse_mla_q8kv8_prefill_fwd supports d_qk=512/576, got {d_q
• sparse_mla_q8kv8_prefill_fwd only supports d_v=512, got {d_v

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7f528da0d20d542c. Report an issue: GitHub.