sgl-project/sglang · error · RuntimeError

SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models

Error message

SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models (e.g. GQA, MHA). MLA models should not set this flag.

What it means

Configuration guard in DisaggregationPrefillController.__init__: the env flag SGLANG_DISAGG_STAGING_BUFFER enables a page-aligned staging grid for KV sends that is only implemented for non-MLA attention backends (GQA/MHA). MLA models have a different KV layout so the flag is rejected at startup.

Source

Thrown at python/sglang/srt/disaggregation/prefill.py:161

        self.is_mla_backend = is_mla_backend(token_to_kv_pool)
        self.metadata_buffers = metadata_buffers
        self.req_to_metadata_buffer_idx_allocator = req_to_metadata_buffer_idx_allocator
        self.tp_rank = tp_rank
        self.tp_size = tp_size
        self.pp_rank = pp_rank
        self.pp_size = pp_size
        self.gpu_id = gpu_id
        self.bootstrap_port = bootstrap_port
        self.queue: List[Req] = []
        self.gloo_group = gloo_group
        self.scheduler = scheduler
        self.max_total_num_tokens = (
            self.scheduler.tp_worker.model_runner.effective_max_total_num_tokens
        )
        self.transfer_backend = transfer_backend
        if envs.SGLANG_DISAGG_STAGING_BUFFER.get():
            if self.is_mla_backend:
                raise RuntimeError(
                    "SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models "
                    "(e.g. GQA, MHA). MLA models should not set this flag."
                )
            page_size = self.scheduler.token_to_kv_pool_allocator.page_size
            # Same source as send_kv_chunk's staging grid below, so validation
            # and the grid cannot disagree after a post-publish override.
            chunked_prefill_size = get_schedule().chunked_prefill_size
            cps = chunked_prefill_size or 8192
            # Staging slices each send into a fixed page-aligned grid, so an
            # unbounded (-1) or non-page-aligned chunk size has no valid grid.
            if cps <= 0 or cps % page_size != 0:
                raise RuntimeError(
                    f"SGLANG_DISAGG_STAGING_BUFFER requires a positive "
                    f"chunked_prefill_size that is a multiple of page_size "
                    f"({page_size}); got {chunked_prefill_size}."
                )
            if self.pp_size > 1 and self.transfer_backend != TransferBackend.MOONCAKE:
                raise RuntimeError(

View on GitHub (pinned to 0132848349)

Solutions

1. Unset SGLANG_DISAGG_STAGING_BUFFER for MLA models (remove it from the launch env / .env).
2. If staging sends are required, use a GQA/MHA model where the flag is supported.
3. Track upstream SGLang for MLA staging-buffer support instead of forcing the flag.

Example fix

# before
export SGLANG_DISAGG_STAGING_BUFFER=1
python -m sglang.launch_server --model deepseek-ai/DeepSeek-V3 --disaggregation-prefill ...
# after (MLA: do not set the flag)
unset SGLANG_DISAGG_STAGING_BUFFER
python -m sglang.launch_server --model deepseek-ai/DeepSeek-V3 --disaggregation-prefill ...

# before launching
import os, subprocess
is_mla = model_uses_mla(MODEL_PATH)  # check config attention architecture
if os.environ.get('SGLANG_DISAGG_STAGING_BUFFER') and is_mla:
    del os.environ['SGLANG_DISAGG_STAGING_BUFFER']

When it happens

Trigger: Setting SGLANG_DISAGG_STAGING_BUFFER=1 (or true) while launching a disaggregated prefill server with an MLA model (e.g. DeepSeek-V2/V3 style attention), making scheduler.tp_worker's model runner report is_mla_backend true.

Common situations: Enabling the staging-buffer optimization to reduce NIXL/Mooncake transfer overhead and then running DeepSeek or another MLA-architecture model; or switching models in a launch script that already exports the env var.

Related errors

• return_sampling_mask with disaggregation requires SGLANG_DIS
• TensorRT-LLM MLA backend only supports kv-cache-dtype of fp8
• tokenspeed_mla backend requires kv-cache-dtype=fp8_e4m3, got
• CuteDSL MLA backend only supports kv-cache-dtype of fp8_e4m3
• SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/956e9d61ef0ce663. Report an issue: GitHub.