sgl-project/sglang · error · RuntimeError

SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models

Error message

SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models (e.g. GQA, MHA). MLA models should not set this flag.

What it means

DecodePreallocQueue.__init__ reads SGLANG_DISAGG_STAGING_BUFFER and hard-rejects it for MLA backends. The staging-buffer path was built for non-MLA (GQA/MHA) KV layouts; MLA's compressed latent does not fit the staging buffer scheme, so the flag is treated as a user error rather than silently ignored.

Source

Thrown at python/sglang/srt/disaggregation/decode.py:389

        self.pending_reqs: List[DecodeRequest] = []
        # In-flight authoritative room -> DP-rank lookups, consumed below.
        self._prefill_dp_rank_queries: Dict[
            str, Tuple[Tuple[int, ...], Future[Dict[str, int]]]
        ] = {}
        self._ensure_retry_count: Dict[str, int] = {}
        self._max_ensure_retries: int = 15  # scheduling cycles
        self._ensure_last_attempt_time: Dict[str, float] = {}
        self._ensure_retry_interval: float = 1.0  # seconds
        # Retracted requests staged for rebootstrap while generation is paused.
        # Enqueued into ``self.queue`` only on ``continue_generation`` so the
        # prefix KV is recomputed under the post-retract (updated) weights.
        # NOTE: requests held here are not reachable by ``/abort_request``; to
        # support aborting them we would need an additional fix in the
        # scheduler. In practice this shouldn't arise in the RL scenario.
        self.held_rebootstrap_reqs: List[Req] = []
        self.enable_staging = envs.SGLANG_DISAGG_STAGING_BUFFER.get()
        if self.enable_staging and self.is_mla_backend:
            raise RuntimeError(
                "SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models "
                "(e.g. GQA, MHA). MLA models should not set this flag."
            )
        self.kv_manager = self._init_kv_manager()
        if self.enable_staging:
            self.transfer_queue._init_staging_handler(self.kv_manager)

        if (
            self.scheduler.tp_worker.is_hybrid_swa
            and not self._uses_swa_tail_prealloc()
        ):
            # Fallback for SWA allocators that still allocate the SWA pool at
            # full prompt length.
            self.max_total_num_tokens = min(
                self.max_total_num_tokens,
                self.scheduler.tp_worker.model_runner.swa_max_total_num_tokens,
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Unset SGLANG_DISAGG_STAGING_BUFFER (or set to 0) for MLA models
2. For MLA models, use the regular (non-staging) PD transfer path
3. If you need staging-like behavior for MLA, track upstream support rather than forcing the flag

Example fix

# before
export SGLANG_DISAGG_STAGING_BUFFER=1  # with DeepSeek (MLA)
# after
unset SGLANG_DISAGG_STAGING_BUFFER

if envs.SGLANG_DISAGG_STAGING_BUFFER.get():
    assert not is_mla_model(model_path), 'staging buffer unsupported for MLA'

def staging_supported(model_backend: str) -> bool:
    return model_backend not in ('mla', 'hybrid_mla')

When it happens

Trigger: Launching a decode server with SGLANG_DISAGG_STAGING_BUFFER=1 (or true) while the model resolves to an MLA backend (DeepSeek V2/V3, etc.), i.e. self.is_mla_backend is True.

Common situations: Copying staging flags from a Qwen/Llama PD deployment to a DeepSeek deployment; experimenting with the staging optimization without reading its model-class restriction.

Related errors

• PD decode DCP requires an MLA or hybrid-MLA KV pool.
• Staging is enabled but kv_manager._staging_ctx.allocator is
• SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models
• unknown q-prep variant {variant!r} (SGLANG_OPT_Q8KV8_QPREP_V
• Unexpected compressed-MLA dst_kv_ptrs length {len(dst_kv_ptr

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4bba4cb98a2a3027. Report an issue: GitHub.