sgl-project/sglang · error · ValueError

HiRadixCache only supports MHA, MLA, DSA, and MSA models

Error message

HiRadixCache only supports MHA, MLA, DSA, and MSA models

What it means

HiRadixCache.__init__ throws this ValueError when the model's attention architecture is not one of MHA, MLA, DSA, or MSA, because HiRadixCache only knows how to lay out/transfer KV for those layouts (it builds the allocator with hicache_mem_layout, dcp_size/rank etc. from the architecture branch). Any other attention variant (e.g. sliding-window-only, cross-attention, or a new arch flag) falls into the else branch and is rejected.

Source

Thrown at python/sglang/srt/mem_cache/hiradix_cache.py:121

        elif isinstance(self.kv_cache, MiniMaxSparseKVPool):
            # Filled by attach_hybrid_minimax_sparse_pool_to_hiradix_cache.
            self.token_to_kv_pool_host = None
        elif isinstance(self.kv_cache, MLATokenToKVPool):
            from sglang.srt.runtime_context import get_parallel

            _parallel = get_parallel()
            self.token_to_kv_pool_host = MLATokenToKVPoolHost(
                self.kv_cache,
                get_memory().hicache_ratio,
                get_memory().hicache_size,
                self.page_size,
                get_memory().hicache_mem_layout,
                allocator_type=allocator_type,
                dcp_size=_parallel.attn_dcp_size,
                dcp_rank=_parallel.attn_dcp_rank,
            )
        else:
            raise ValueError("HiRadixCache only supports MHA, MLA, DSA, and MSA models")

        self.tp_group = params.tp_cache_group
        self.attn_cp_group = params.attn_cp_cache_group
        self.attn_tp_group = params.attn_tp_cache_group
        self.pp_group = params.pp_cache_group
        self.tp_world_size = torch.distributed.get_world_size(group=self.tp_group)
        self.pp_rank = params.pp_rank
        self.pp_size = params.pp_size
        self.enable_storage = get_memory().hicache_storage_backend is not None
        self.enable_storage_metrics = self.enable_storage and params.enable_metrics
        self.extra_metric_labels = get_observability().extra_metric_labels

        (
            extra_config,
            prefetch_threshold,
            prefetch_timeout_config,
            hicache_storage_pass_prefix_keys,
        ) = self._parse_storage_backend_extra_config(

View on GitHub (pinned to 0132848349)

Solutions

1. Disable hierarchical cache for this model (drop --enable-hierarchical-cache / hi-radix flags) and run with the regular RadixCache
2. Check model support: confirm the model's attention type is one of MHA, MLA, DSA, MSA before enabling HiRadixCache
3. Switch to a supported model variant (e.g. a standard MHA/MLA checkpoint) if hierarchical caching is required
4. If you maintain the model, add/verify the architecture branch so its pool type is recognized by HiRadixCache

Example fix

# before
launch_server(..., model_path="gemma-3", enable_hierarchical_cache=True)

# after: unsupported arch, use plain radix cache
launch_server(..., model_path="gemma-3")

from sglang.srt.mem_cache.memory_pool import MHA, MLA, DSA, MSA  # arch pool types
supported = is_mha(model) or is_mla(model) or is_dsa(model) or is_msa(model)
if want_hiradix and not supported:
    logger.warning("HiRadixCache unsupported for this arch; falling back to RadixCache")

def supports_hiradix(model_config) -> bool:
    return model_config.attention_arch in ("MHA", "MLA", "DSA", "MSA")

When it happens

Trigger: Constructing HiRadixCache (e.g. enabling --enable-hierarchical-cache / hi-radix cache) with a model whose attention backend/arch is not MHA/MLA/DSA/MSA — the if/elif chain over architecture types falls through to the else.

Common situations: Enabling HiCache/HiRadixCache on a sliding-window or hybrid-attention model (e.g. Gemma, hybrid linear-attention models) that has no supported hi-cache layout; upgrading SGLang where new arch flags changed the branch conditions; custom model implementations that don't subclass the supported attention pool types.

Related errors

• /v1/models ${response.status}
• Unsupported msgpack byte ${b}
• This browser does not support gzip stream decoding
• Unsupported model type: {model_type}
• Action endpoint is not implemented for {sampling_params_cls.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c6b5db2a29b6f946. Report an issue: GitHub.