sgl-project/sglang · error · AttributeError

Unsupported KV cache type {type(kvcache).name}: expected

Error message

Unsupported KV cache type {type(kvcache).__name__}: expected kv_buffer (MLA/NSA) or k_buffer/v_buffer (MHA).

What it means

The FlexKV connector's __init__ inspects the KV cache object to extract per-layer buffers: it accepts kv_buffer (MLA/NSA, where K and V share one per-layer tensor) or k_buffer + v_buffer (MHA, concatenated layer-first). If the cache object exposes neither attribute set, it raises AttributeError because there is no supported way to map the cache layout onto FlexKV storage.

Source

Thrown at python/sglang/srt/mem_cache/storage/flexkv/flexkv_connector.py:150

            if aligned != orig:
                logger.info(
                    "[FlexKV] Block count MIN alignment '%s': %d -> %d",
                    attr,
                    orig,
                    aligned,
                )
            setattr(self.cache_config, attr, aligned)

        # 4. Extract MLA/MHA KV buffers + optional indexer buffers.
        indexer_buffers = getattr(kvcache, "index_k_with_scale_buffer", None)
        if hasattr(kvcache, "kv_buffer"):
            # MLA: K and V share the same buffer (per-layer tensor).
            kv_caches = list(kvcache.kv_buffer)
        elif hasattr(kvcache, "k_buffer"):
            # MHA: K buffers concatenated with V buffers, layer-first.
            kv_caches = list(kvcache.k_buffer) + list(kvcache.v_buffer)
        else:
            raise AttributeError(
                f"Unsupported KV cache type {type(kvcache).__name__}: "
                f"expected kv_buffer (MLA/NSA) or k_buffer/v_buffer (MHA)."
            )
        self._kvcache = kvcache

        # 5. On multi-node setups, every node beyond node 0 needs a
        # TransferManagerOnRemote process (FlexKV side) before any rank
        # on that node can register GPU buffers.
        self._remote_process = None
        if (
            self.model_config.nnodes > 1
            and self.rank_info.node_rank > 0
            and self.rank_info.local_rank == 0
        ):
            self._remote_process = TransferManagerOnRemote.create_process(
                master_host=self.model_config.master_host,
                master_ports=self.model_config.master_ports,
            )

View on GitHub (pinned to 0132848349)

Solutions

Check type(kvcache) and its attributes; if it stores a single combined tensor per layer under a different name, expose it as kv_buffer (MLA-style) before constructing the connector
Switch to an attention backend whose pool uses k_buffer/v_buffer (MHA) or kv_buffer (MLA/NSA)
Do not enable the FlexKV storage backend for hybrid/linear-attention models that have no per-layer K/V buffers

Example fix

// before
connector = FlexKVConnector(kvcache=hybrid_pool)  # hybrid_pool has kv_cache attr only

// after
# expose MLA-style shared buffer
hybrid_pool.kv_buffer = hybrid_pool.kv_cache
connector = FlexKVConnector(kvcache=hybrid_pool)

Defensive patterns

Strategy: type-guard

Validate before calling

def has_supported_kv_buffers(kvcache) -> bool:
    return hasattr(kvcache, 'kv_buffer') or (
        hasattr(kvcache, 'k_buffer') and hasattr(kvcache, 'v_buffer')
    )

if not has_supported_kv_buffers(pool):
    raise SkipFlexKVSetup(f'unsupported pool {type(pool).__name__}')

Type guard

from typing import Protocol

class MLAStyleCache(Protocol):
    kv_buffer: list

class MHAStyleCache(Protocol):
    k_buffer: list
    v_buffer: list

def is_supported_cache(c: object) -> bool:
    return isinstance(c, (MLAStyleCache, MHAStyleCache))

Try / catch

try:
    connector = FlexKVConnector(kvcache=pool, ...)
except AttributeError as e:
    if 'Unsupported KV cache type' in str(e):
        logger.warning('FlexKV disabled: %s', e)
        connector = None  # fall back to local-only cache
    else:
        raise

Prevention

Assert the buffer attributes in integration tests for every attention backend you run with FlexKV
Keep a compatibility matrix of backend pool types vs FlexKV support and check it at startup

When it happens

Trigger: Passing a custom or new attention-backend KV cache pool to FlexKVConnector that implements neither kv_buffer nor k_buffer/v_buffer (e.g. a hybrid/Mamba pool, a renamed buffer like kv_cache, or a pool with only a single flattened cache tensor).

Common situations: Upgrading SGLang where a backend renamed its buffers; using a non-MLA/MHA architecture (linear attention, hybrid models) with the FlexKV storage backend; mocking the cache incompletely in tests.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2311d20998eae94a. Report an issue: GitHub.