{"record":{"id":"2311d20998eae94a","repo":"sgl-project/sglang","slug":"unsupported-kv-cache-type-type-kvcache-name","errorCode":null,"errorMessage":"Unsupported KV cache type {type(kvcache).__name__}: expected kv_buffer (MLA/NSA) or k_buffer/v_buffer (MHA).","messagePattern":"Unsupported KV cache type (.+?): expected kv_buffer \\(MLA/NSA\\) or k_buffer/v_buffer \\(MHA\\)\\.","errorType":"validation","errorClass":"AttributeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/storage/flexkv/flexkv_connector.py","lineNumber":150,"sourceCode":"            if aligned != orig:\n                logger.info(\n                    \"[FlexKV] Block count MIN alignment '%s': %d -> %d\",\n                    attr,\n                    orig,\n                    aligned,\n                )\n            setattr(self.cache_config, attr, aligned)\n\n        # 4. Extract MLA/MHA KV buffers + optional indexer buffers.\n        indexer_buffers = getattr(kvcache, \"index_k_with_scale_buffer\", None)\n        if hasattr(kvcache, \"kv_buffer\"):\n            # MLA: K and V share the same buffer (per-layer tensor).\n            kv_caches = list(kvcache.kv_buffer)\n        elif hasattr(kvcache, \"k_buffer\"):\n            # MHA: K buffers concatenated with V buffers, layer-first.\n            kv_caches = list(kvcache.k_buffer) + list(kvcache.v_buffer)\n        else:\n            raise AttributeError(\n                f\"Unsupported KV cache type {type(kvcache).__name__}: \"\n                f\"expected kv_buffer (MLA/NSA) or k_buffer/v_buffer (MHA).\"\n            )\n        self._kvcache = kvcache\n\n        # 5. On multi-node setups, every node beyond node 0 needs a\n        # TransferManagerOnRemote process (FlexKV side) before any rank\n        # on that node can register GPU buffers.\n        self._remote_process = None\n        if (\n            self.model_config.nnodes > 1\n            and self.rank_info.node_rank > 0\n            and self.rank_info.local_rank == 0\n        ):\n            self._remote_process = TransferManagerOnRemote.create_process(\n                master_host=self.model_config.master_host,\n                master_ports=self.model_config.master_ports,\n            )","sourceCodeStart":132,"sourceCodeEnd":168,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/storage/flexkv/flexkv_connector.py#L132-L168","documentation":"The FlexKV connector's __init__ inspects the KV cache object to extract per-layer buffers: it accepts kv_buffer (MLA/NSA, where K and V share one per-layer tensor) or k_buffer + v_buffer (MHA, concatenated layer-first). If the cache object exposes neither attribute set, it raises AttributeError because there is no supported way to map the cache layout onto FlexKV storage.","triggerScenarios":"Passing a custom or new attention-backend KV cache pool to FlexKVConnector that implements neither kv_buffer nor k_buffer/v_buffer (e.g. a hybrid/Mamba pool, a renamed buffer like kv_cache, or a pool with only a single flattened cache tensor).","commonSituations":"Upgrading SGLang where a backend renamed its buffers; using a non-MLA/MHA architecture (linear attention, hybrid models) with the FlexKV storage backend; mocking the cache incompletely in tests.","solutions":["Check type(kvcache) and its attributes; if it stores a single combined tensor per layer under a different name, expose it as kv_buffer (MLA-style) before constructing the connector","Switch to an attention backend whose pool uses k_buffer/v_buffer (MHA) or kv_buffer (MLA/NSA)","Do not enable the FlexKV storage backend for hybrid/linear-attention models that have no per-layer K/V buffers"],"exampleFix":"// before\nconnector = FlexKVConnector(kvcache=hybrid_pool)  # hybrid_pool has kv_cache attr only\n\n// after\n# expose MLA-style shared buffer\nhybrid_pool.kv_buffer = hybrid_pool.kv_cache\nconnector = FlexKVConnector(kvcache=hybrid_pool)","handlingStrategy":"type-guard","validationCode":"def has_supported_kv_buffers(kvcache) -> bool:\n    return hasattr(kvcache, 'kv_buffer') or (\n        hasattr(kvcache, 'k_buffer') and hasattr(kvcache, 'v_buffer')\n    )\n\nif not has_supported_kv_buffers(pool):\n    raise SkipFlexKVSetup(f'unsupported pool {type(pool).__name__}')","typeGuard":"from typing import Protocol\n\nclass MLAStyleCache(Protocol):\n    kv_buffer: list\n\nclass MHAStyleCache(Protocol):\n    k_buffer: list\n    v_buffer: list\n\ndef is_supported_cache(c: object) -> bool:\n    return isinstance(c, (MLAStyleCache, MHAStyleCache))","tryCatchPattern":"try:\n    connector = FlexKVConnector(kvcache=pool, ...)\nexcept AttributeError as e:\n    if 'Unsupported KV cache type' in str(e):\n        logger.warning('FlexKV disabled: %s', e)\n        connector = None  # fall back to local-only cache\n    else:\n        raise","preventionTips":["Assert the buffer attributes in integration tests for every attention backend you run with FlexKV","Keep a compatibility matrix of backend pool types vs FlexKV support and check it at startup"],"tags":["flexkv","kv-cache","attributeerror","attention-backend"],"backgroundTag":"unsupported-object-shape","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}