sgl-project/sglang · error · AttributeError

Unsupported KV cache type

Error message

Unsupported KV cache type {type(kvcache).__name__}: expected kv_buffer (MLA/NSA) or k_buffer/v_buffer (MHA).

What it means

The FlexKV connector's __init__ inspects the KV cache object to extract per-layer buffers: it accepts kv_buffer (MLA/NSA, where K and V share one per-layer tensor) or k_buffer + v_buffer (MHA, concatenated layer-first). If the cache object exposes neither attribute set, it raises AttributeError because there is no supported way to map the cache layout onto FlexKV storage.

Solutions

  1. Check type(kvcache) and its attributes; if it stores a single combined tensor per layer under a different name, expose it as kv_buffer (MLA-style) before constructing the connector
  2. Switch to an attention backend whose pool uses k_buffer/v_buffer (MHA) or kv_buffer (MLA/NSA)
  3. Do not enable the FlexKV storage backend for hybrid/linear-attention models that have no per-layer K/V buffers

Example fix

// before
connector = FlexKVConnector(kvcache=hybrid_pool)  # hybrid_pool has kv_cache attr only

// after
# expose MLA-style shared buffer
hybrid_pool.kv_buffer = hybrid_pool.kv_cache
connector = FlexKVConnector(kvcache=hybrid_pool)
Defensive patterns

Strategy: type-guard

Validate before calling

def has_supported_kv_buffers(kvcache) -> bool:
    return hasattr(kvcache, 'kv_buffer') or (
        hasattr(kvcache, 'k_buffer') and hasattr(kvcache, 'v_buffer')
    )

if not has_supported_kv_buffers(pool):
    raise SkipFlexKVSetup(f'unsupported pool {type(pool).__name__}')

Type guard

from typing import Protocol

class MLAStyleCache(Protocol):
    kv_buffer: list

class MHAStyleCache(Protocol):
    k_buffer: list
    v_buffer: list

def is_supported_cache(c: object) -> bool:
    return isinstance(c, (MLAStyleCache, MHAStyleCache))

Try / catch

try:
    connector = FlexKVConnector(kvcache=pool, ...)
except AttributeError as e:
    if 'Unsupported KV cache type' in str(e):
        logger.warning('FlexKV disabled: %s', e)
        connector = None  # fall back to local-only cache
    else:
        raise

Prevention

When it happens

Trigger: Passing a custom or new attention-backend KV cache pool to FlexKVConnector that implements neither kv_buffer nor k_buffer/v_buffer (e.g. a hybrid/Mamba pool, a renamed buffer like kv_cache, or a pool with only a single flattened cache tensor).

Common situations: Upgrading SGLang where a backend renamed its buffers; using a non-MLA/MHA architecture (linear attention, hybrid models) with the FlexKV storage backend; mocking the cache incompletely in tests.

Related errors


AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2311d20998eae94a. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/mem_cache/storage/flexkv/flexkv_connector.py:150

            if aligned != orig:
                logger.info(
                    "[FlexKV] Block count MIN alignment '%s': %d -> %d",
                    attr,
                    orig,
                    aligned,
                )
            setattr(self.cache_config, attr, aligned)

        # 4. Extract MLA/MHA KV buffers + optional indexer buffers.
        indexer_buffers = getattr(kvcache, "index_k_with_scale_buffer", None)
        if hasattr(kvcache, "kv_buffer"):
            # MLA: K and V share the same buffer (per-layer tensor).
            kv_caches = list(kvcache.kv_buffer)
        elif hasattr(kvcache, "k_buffer"):
            # MHA: K buffers concatenated with V buffers, layer-first.
            kv_caches = list(kvcache.k_buffer) + list(kvcache.v_buffer)
        else:
            raise AttributeError(
                f"Unsupported KV cache type {type(kvcache).__name__}: "
                f"expected kv_buffer (MLA/NSA) or k_buffer/v_buffer (MHA)."
            )
        self._kvcache = kvcache

        # 5. On multi-node setups, every node beyond node 0 needs a
        # TransferManagerOnRemote process (FlexKV side) before any rank
        # on that node can register GPU buffers.
        self._remote_process = None
        if (
            self.model_config.nnodes > 1
            and self.rank_info.node_rank > 0
            and self.rank_info.local_rank == 0
        ):
            self._remote_process = TransferManagerOnRemote.create_process(
                master_host=self.model_config.master_host,
                master_ports=self.model_config.master_ports,
            )

View on GitHub (pinned to 0132848349)