sgl-project/sglang · error · ValueError

--enable-unified-memory with PD disaggregation does not supp

Error message

--enable-unified-memory with PD disaggregation does not support hybrid-SWA models yet (no whole-envelope transfer scheme for the SWA sub-pool). Drop --enable-unified-memory or run without PD.

What it means

Raised during KV pool initialization when --enable-unified-memory is combined with prefill/decode (PD) disaggregation on a hybrid sliding-window-attention (SWA) model. There is no whole-envelope transfer scheme for the SWA sub-pool, so the unified pool cannot be transferred between prefill and decode workers.

Source

Thrown at python/sglang/srt/mem_cache/kv_cache_configurator.py:402

        # (req_to_token_pool is None); supports hybrid Mamba and hybrid SWA (not DSV4).
        if get_memory().enable_unified_memory and req_to_token_pool is None:
            pd_enabled = get_disagg().disaggregation_mode != "null"
            if self.mambaish_config is not None:
                if pd_enabled and not self.use_mla_backend:
                    raise ValueError(
                        "--enable-unified-memory with PD disaggregation "
                        "currently supports only MLA hybrid-Mamba models "
                        "(e.g. kimi-linear); this model uses the MHA full-"
                        "attention pool. Drop --enable-unified-memory or run "
                        "without PD disaggregation."
                    )
                bundle = self._init_unified_mamba_pools(
                    max_num_reqs=sizes.max_running_requests,
                    max_total_num_tokens=sizes.max_total_num_tokens,
                )
            elif self.is_hybrid_swa and not is_deepseek_v4(self.model_config.hf_config):
                if pd_enabled:
                    raise ValueError(
                        "--enable-unified-memory with PD disaggregation does "
                        "not support hybrid-SWA models yet (no whole-envelope "
                        "transfer scheme for the SWA sub-pool). Drop "
                        "--enable-unified-memory or run without PD."
                    )
                bundle = self._init_unified_swa_pools(
                    max_num_reqs=sizes.max_running_requests,
                    full_max_total_num_tokens=sizes.full_max_total_num_tokens,
                    swa_max_total_num_tokens=sizes.swa_max_total_num_tokens,
                )
            else:
                # Fail loud, not silently fall through to the normal pools (which would
                # leave the flag a no-op). The feature replaces the HYBRID pools only.
                raise ValueError(
                    "--enable-unified-memory only supports hybrid Mamba and "
                    "hybrid sliding-window-attention models (DeepSeek-V4 excluded); "
                    f"the current model ({self.model_config.hf_config.architectures}) "
                    "is neither, so the unified memory pool cannot be built. Drop "

View on GitHub (pinned to 0132848349)

Solutions

1. Remove --enable-unified-memory from the server args
2. Run the same model without PD disaggregation (single-node / non-PD topology)
3. Wait for a release that implements SWA sub-pool whole-envelope transfer

Example fix

# before
python -m sglang.launch_server --model hybrid-swa-model --enable-unified-memory --disagg-pd ...
# after
python -m sglang.launch_server --model hybrid-swa-model --disagg-pd ...

from sglang.srt.utils import is_hybrid_swa  # conceptual
pd_enabled = server_args.disaggregation_mode in ("prefill", "decode")
if server_args.enable_unified_memory and pd_enabled and is_hybrid_swa(model_config):
    raise SystemExit("unified-memory + PD unsupported for hybrid-SWA; dropping --enable-unified-memory")

When it happens

Trigger: Boot a server with --enable-unified-memory on a hybrid-SWA architecture while PD disaggregation is enabled; _init_pools raises before any pool is created.

Common situations: Trying the new unified memory pool on a hybrid-SWA model in a PD-disaggregated deployment before that combination was implemented.

Related errors

• --enable-unified-memory only supports hybrid Mamba and hybri
• Speculative decoding with --enable-unified-memory is only su
• --prefill-only-disable-kv-cache expected NoOpMHATokenToKVPoo
• v_cache must be provided
• k_cache can only be None when only_qv=True

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/720a9ac0c8808028. Report an issue: GitHub.