sgl-project/sglang · error · ValueError

--enable-unified-memory only supports hybrid Mamba and hybri

Error message

--enable-unified-memory only supports hybrid Mamba and hybrid sliding-window-attention models (DeepSeek-V4 excluded); the current model ({self.model_config.hf_config.architectures}) is neither, so the unified memory pool cannot be built. Drop --enable-unified-memory for this model.

What it means

--enable-unified-memory replaces only the HYBRID pools and supports hybrid Mamba and hybrid SWA models (DeepSeek-V4 excluded). This error fires when the flag is passed for any other architecture, refusing to silently fall through to the normal pools where the flag would be a no-op.

Source

Thrown at python/sglang/srt/mem_cache/kv_cache_configurator.py:416

                    max_total_num_tokens=sizes.max_total_num_tokens,
                )
            elif self.is_hybrid_swa and not is_deepseek_v4(self.model_config.hf_config):
                if pd_enabled:
                    raise ValueError(
                        "--enable-unified-memory with PD disaggregation does "
                        "not support hybrid-SWA models yet (no whole-envelope "
                        "transfer scheme for the SWA sub-pool). Drop "
                        "--enable-unified-memory or run without PD."
                    )
                bundle = self._init_unified_swa_pools(
                    max_num_reqs=sizes.max_running_requests,
                    full_max_total_num_tokens=sizes.full_max_total_num_tokens,
                    swa_max_total_num_tokens=sizes.swa_max_total_num_tokens,
                )
            else:
                # Fail loud, not silently fall through to the normal pools (which would
                # leave the flag a no-op). The feature replaces the HYBRID pools only.
                raise ValueError(
                    "--enable-unified-memory only supports hybrid Mamba and "
                    "hybrid sliding-window-attention models (DeepSeek-V4 excluded); "
                    f"the current model ({self.model_config.hf_config.architectures}) "
                    "is neither, so the unified memory pool cannot be built. Drop "
                    "--enable-unified-memory for this model."
                )
            return _InitializedPools(
                req_to_token_pool=bundle.req_to_token_pool,
                token_to_kv_pool=bundle.token_to_kv_pool,
                token_to_kv_pool_allocator=bundle.token_to_kv_pool_allocator,
                unified_memory_pool=bundle.unified_memory_pool,
            )

        # The unified allocator hands out VIRTUAL token ids from the whole
        # virtual space (> max_total_num_tokens); the direct-indexed draft
        # pool must be sized by that space.
        draft_virtual_id_space: Optional[int] = None
        if self.is_draft_worker and token_to_kv_pool_allocator is not None:

View on GitHub (pinned to 0132848349)

Solutions

  1. Drop --enable-unified-memory for this model
  2. Switch to a supported hybrid Mamba or hybrid SWA model if you need the feature

Example fix

# before
python -m sglang.launch_server --model llama-... --enable-unified-memory
# after
python -m sglang.launch_server --model llama-...
Defensive patterns

Strategy: validation

Validate before calling

arch = model_config.hf_config.architectures
supported = is_hybrid_mamba(model_config) or (is_hybrid_swa(model_config) and not is_deepseek_v4(model_config.hf_config))
if server_args.enable_unified_memory and not supported:
    server_args.enable_unified_memory = False  # or abort

Prevention

When it happens

Trigger: Pass --enable-unified-memory with a model whose architectures are neither hybrid-Mamba nor hybrid-SWA (or is DeepSeek-V4); _init_pools hits the else branch and raises.

Common situations: Copy-pasting a launch command with the new flag onto a standard dense/LLM model, or onto DeepSeek-V4.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ce5cbefa16d6fad4. Report an issue: GitHub.