sgl-project/sglang · error · RuntimeError

SGLANG_DISAGG_STAGING_BUFFER with pp_size > 1 is only suppor

Error message

SGLANG_DISAGG_STAGING_BUFFER with pp_size > 1 is only supported by Mooncake.

What it means

With SGLANG_DISAGG_STAGING_BUFFER enabled and pipeline parallelism (pp_size > 1), only the Mooncake transfer backend implements the staging buffer correctly across pipeline stages; NIXL is rejected at startup.

Source

Thrown at python/sglang/srt/disaggregation/prefill.py:179

                raise RuntimeError(
                    "SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models "
                    "(e.g. GQA, MHA). MLA models should not set this flag."
                )
            page_size = self.scheduler.token_to_kv_pool_allocator.page_size
            # Same source as send_kv_chunk's staging grid below, so validation
            # and the grid cannot disagree after a post-publish override.
            chunked_prefill_size = get_schedule().chunked_prefill_size
            cps = chunked_prefill_size or 8192
            # Staging slices each send into a fixed page-aligned grid, so an
            # unbounded (-1) or non-page-aligned chunk size has no valid grid.
            if cps <= 0 or cps % page_size != 0:
                raise RuntimeError(
                    f"SGLANG_DISAGG_STAGING_BUFFER requires a positive "
                    f"chunked_prefill_size that is a multiple of page_size "
                    f"({page_size}); got {chunked_prefill_size}."
                )
            if self.pp_size > 1 and self.transfer_backend != TransferBackend.MOONCAKE:
                raise RuntimeError(
                    "SGLANG_DISAGG_STAGING_BUFFER with pp_size > 1 is only "
                    "supported by Mooncake."
                )
            if get_parallel().enable_prefill_context_parallel:
                # CP rewrites index_slice per rank, breaking the chunk grid.
                raise RuntimeError(
                    "SGLANG_DISAGG_STAGING_BUFFER does not support "
                    "prefill context parallelism."
                )
        self.kv_manager = self._init_kv_manager()

    def _init_kv_manager(self) -> CommonKVManager:
        kv_args_class = get_kv_class(self.transfer_backend, KVClassType.KVARGS)
        kv_args = kv_args_class()
        kv_args.engine_rank = self.tp_rank
        kv_args.pp_rank = self.pp_rank
        kv_args.system_dp_rank = self.scheduler.ps.dp_rank
        kv_args.kv_cache_dtype_str = (

View on GitHub (pinned to 0132848349)

Solutions

1. Switch to --disaggregation-transfer-backend mooncake when pp_size > 1 with staging buffer.
2. Or reduce --pipeline-parallel-size to 1.
3. Or unset SGLANG_DISAGG_STAGING_BUFFER to keep NIXL with PP.

Example fix

# before
export SGLANG_DISAGG_STAGING_BUFFER=1
python -m sglang.launch_server --disaggregation-transfer-backend nixl --pipeline-parallel-size 2 ...
# after
export SGLANG_DISAGG_STAGING_BUFFER=1
python -m sglang.launch_server --disaggregation-transfer-backend mooncake --pipeline-parallel-size 2 ...

if envs.SGLANG_DISAGG_STAGING_BUFFER.get() and pp_size > 1:
    assert transfer_backend == TransferBackend.MOONCAKE, 'staging buffer + PP requires mooncake'

When it happens

Trigger: SGLANG_DISAGG_STAGING_BUFFER=1 plus --pipeline-parallel-size > 1 while --disaggregation-transfer-backend nixl (or any non-mooncake backend) is configured.

Common situations: Scaling up a PD deployment to multi-stage pipeline parallelism and keeping NIXL as transfer backend with the staging optimization still exported in the launch script.

Related errors

• KVTransferError(self.bootstrap_room, failure_reason)
• NIXL KVSender Exception
• NIXL KVReceiver Exception
• SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models
• SGLANG_DISAGG_STAGING_BUFFER requires a positive chunked_pre

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7d4c459a344d3c97. Report an issue: GitHub.