sgl-project/sglang · error · ValueError

--enable-unified-memory supports monolithic (decode) cuda-gr

Error message

--enable-unified-memory supports monolithic (decode) cuda-graph capture only; disable piecewise prefill capture (e.g. --cuda-graph-backend-prefill=disabled).

What it means

Raised when --enable-unified-memory (DCP/unified memory pool) is combined with piecewise (torch.compile) prefill CUDA-graph capture. Only monolithic decode-graph capture is wired for the unified-memory path; piecewise prefill capture is not, so the combination is rejected at startup.

Source

Thrown at python/sglang/srt/server_args.py:9359

            )
        assert not (cfg.enable_hierarchical_cache or cfg.enable_lmcache), (
            "--enable-unified-memory is not yet compatible with hierarchical / "
            "host-tiered KV cache (--enable-hierarchical-cache / --enable-lmcache): "
            "the unified-memory-pool init wires up no host pools, and its device mamba / "
            "full-attention slots are VIRTUAL — the host-offload path does not "
            "translate them to physical."
        )
        assert cfg.dcp_size == 1, (
            "--enable-unified-memory is not yet compatible with decode context "
            "parallelism (--dcp-size > 1): the pool has no DCP-aware masked write "
            "path (UnifiedMHATokenToKVPool.set_kv_buffer asserts dcp_kv_mask is None), "
            "so a DCP run would boot and then fail on the first KV write."
        )
        # Only monolithic decode cuda-graph capture is wired; piecewise prefill
        # capture is not. Guard when the user opts into it.
        _cg_cfg = cfg.cuda_graph_config
        if _cg_cfg is not None and _cg_cfg.prefill.backend == Backend.TC_PIECEWISE:
            raise ValueError(
                "--enable-unified-memory supports monolithic (decode) "
                "cuda-graph capture only; disable piecewise prefill capture "
                "(e.g. --cuda-graph-backend-prefill=disabled)."
            )
        # The strided-layout Triton requirement is enforced via
        # --enable-page-major-kv-layout (implied by the unified pool in
        # _handle_page_major_kv_layout); the model-family gate is enforced at pool
        # construction in model_runner_kv_cache_mixin._init_pools.

    def _handle_page_major_kv_layout(self):
        # The unified pool stores state in the page-major envelope-strided layout, so
        # enabling it implies --enable-page-major-kv-layout — routing it through the
        # single page-major path + stride-aware Triton asserts (set before the guard).
        cfg = resolving_view(self)
        if cfg.enable_unified_memory:
            self._declare(
                "_handle_page_major_kv_layout",
                enable_page_major_kv_layout=True,

View on GitHub (pinned to 0132848349)

Solutions

Disable prefill capture: --cuda-graph-backend-prefill=disabled
Or keep piecewise capture but drop --enable-unified-memory
Keep decode graph capture on (monolithic) which is supported

Example fix

# before
--enable-unified-memory --cuda-graph-backend-prefill=piecewise
# after
--enable-unified-memory --cuda-graph-backend-prefill=disabled

Defensive patterns

Strategy: validation

Validate before calling

if args.enable_unified_memory:
    assert not (args.cuda_graph_config and args.cuda_graph_config.prefill.backend.name == 'TC_PIECEWISE'), \
        'unified memory requires monolithic graphs; set --cuda-graph-backend-prefill=disabled'

Prevention

When enabling memory pooling flags, audit all cuda-graph flags for compatibility
Default launch scripts to --cuda-graph-backend-prefill=disabled on the unified-memory path

When it happens

Trigger: Passing --enable-unified-memory while cuda_graph_config.prefill.backend == TC_PIECEWISE (explicitly or via default when piecewise prefill capture is enabled).

Common situations: Enabling a global '--cuda-graph-backend piecewise' style flag alongside unified memory; newer SGLang defaulting prefill capture to piecewise; tuning both memory pooling and graph capture simultaneously.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f69dc3b1b1605016. Report an issue: GitHub.