{"record":{"id":"c6b5db2a29b6f946","repo":"sgl-project/sglang","slug":"hiradixcache-only-supports-mha-mla-dsa-and-msa","errorCode":null,"errorMessage":"HiRadixCache only supports MHA, MLA, DSA, and MSA models","messagePattern":"HiRadixCache only supports MHA, MLA, DSA, and MSA models","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/hiradix_cache.py","lineNumber":121,"sourceCode":"        elif isinstance(self.kv_cache, MiniMaxSparseKVPool):\n            # Filled by attach_hybrid_minimax_sparse_pool_to_hiradix_cache.\n            self.token_to_kv_pool_host = None\n        elif isinstance(self.kv_cache, MLATokenToKVPool):\n            from sglang.srt.runtime_context import get_parallel\n\n            _parallel = get_parallel()\n            self.token_to_kv_pool_host = MLATokenToKVPoolHost(\n                self.kv_cache,\n                get_memory().hicache_ratio,\n                get_memory().hicache_size,\n                self.page_size,\n                get_memory().hicache_mem_layout,\n                allocator_type=allocator_type,\n                dcp_size=_parallel.attn_dcp_size,\n                dcp_rank=_parallel.attn_dcp_rank,\n            )\n        else:\n            raise ValueError(\"HiRadixCache only supports MHA, MLA, DSA, and MSA models\")\n\n        self.tp_group = params.tp_cache_group\n        self.attn_cp_group = params.attn_cp_cache_group\n        self.attn_tp_group = params.attn_tp_cache_group\n        self.pp_group = params.pp_cache_group\n        self.tp_world_size = torch.distributed.get_world_size(group=self.tp_group)\n        self.pp_rank = params.pp_rank\n        self.pp_size = params.pp_size\n        self.enable_storage = get_memory().hicache_storage_backend is not None\n        self.enable_storage_metrics = self.enable_storage and params.enable_metrics\n        self.extra_metric_labels = get_observability().extra_metric_labels\n\n        (\n            extra_config,\n            prefetch_threshold,\n            prefetch_timeout_config,\n            hicache_storage_pass_prefix_keys,\n        ) = self._parse_storage_backend_extra_config(","sourceCodeStart":103,"sourceCodeEnd":139,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/hiradix_cache.py#L103-L139","documentation":"HiRadixCache.__init__ throws this ValueError when the model's attention architecture is not one of MHA, MLA, DSA, or MSA, because HiRadixCache only knows how to lay out/transfer KV for those layouts (it builds the allocator with hicache_mem_layout, dcp_size/rank etc. from the architecture branch). Any other attention variant (e.g. sliding-window-only, cross-attention, or a new arch flag) falls into the else branch and is rejected.","triggerScenarios":"Constructing HiRadixCache (e.g. enabling --enable-hierarchical-cache / hi-radix cache) with a model whose attention backend/arch is not MHA/MLA/DSA/MSA — the if/elif chain over architecture types falls through to the else.","commonSituations":"Enabling HiCache/HiRadixCache on a sliding-window or hybrid-attention model (e.g. Gemma, hybrid linear-attention models) that has no supported hi-cache layout; upgrading SGLang where new arch flags changed the branch conditions; custom model implementations that don't subclass the supported attention pool types.","solutions":["Disable hierarchical cache for this model (drop --enable-hierarchical-cache / hi-radix flags) and run with the regular RadixCache","Check model support: confirm the model's attention type is one of MHA, MLA, DSA, MSA before enabling HiRadixCache","Switch to a supported model variant (e.g. a standard MHA/MLA checkpoint) if hierarchical caching is required","If you maintain the model, add/verify the architecture branch so its pool type is recognized by HiRadixCache"],"exampleFix":"# before\nlaunch_server(..., model_path=\"gemma-3\", enable_hierarchical_cache=True)\n\n# after: unsupported arch, use plain radix cache\nlaunch_server(..., model_path=\"gemma-3\")","handlingStrategy":"validation","validationCode":"from sglang.srt.mem_cache.memory_pool import MHA, MLA, DSA, MSA  # arch pool types\nsupported = is_mha(model) or is_mla(model) or is_dsa(model) or is_msa(model)\nif want_hiradix and not supported:\n    logger.warning(\"HiRadixCache unsupported for this arch; falling back to RadixCache\")","typeGuard":"def supports_hiradix(model_config) -> bool:\n    return model_config.attention_arch in (\"MHA\", \"MLA\", \"DSA\", \"MSA\")","tryCatchPattern":null,"preventionTips":["Check the model's attention architecture before passing --enable-hierarchical-cache","Keep a whitelist of tested models when enabling HiRadixCache in fleet configs"],"tags":["hicache","hiradixcache","unsupported-architecture","model-support"],"backgroundTag":"unsupported-architecture-feature-combo","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}