{"record":{"id":"4bba4cb98a2a3027","repo":"sgl-project/sglang","slug":"sglang-disagg-staging-buffer-is-designed-for-non-m","errorCode":null,"errorMessage":"SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models (e.g. GQA, MHA). MLA models should not set this flag.","messagePattern":"SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models \\(e\\.g\\. GQA, MHA\\)\\. MLA models should not set this flag\\.","errorType":"validation","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/disaggregation/decode.py","lineNumber":389,"sourceCode":"        self.pending_reqs: List[DecodeRequest] = []\n        # In-flight authoritative room -> DP-rank lookups, consumed below.\n        self._prefill_dp_rank_queries: Dict[\n            str, Tuple[Tuple[int, ...], Future[Dict[str, int]]]\n        ] = {}\n        self._ensure_retry_count: Dict[str, int] = {}\n        self._max_ensure_retries: int = 15  # scheduling cycles\n        self._ensure_last_attempt_time: Dict[str, float] = {}\n        self._ensure_retry_interval: float = 1.0  # seconds\n        # Retracted requests staged for rebootstrap while generation is paused.\n        # Enqueued into ``self.queue`` only on ``continue_generation`` so the\n        # prefix KV is recomputed under the post-retract (updated) weights.\n        # NOTE: requests held here are not reachable by ``/abort_request``; to\n        # support aborting them we would need an additional fix in the\n        # scheduler. In practice this shouldn't arise in the RL scenario.\n        self.held_rebootstrap_reqs: List[Req] = []\n        self.enable_staging = envs.SGLANG_DISAGG_STAGING_BUFFER.get()\n        if self.enable_staging and self.is_mla_backend:\n            raise RuntimeError(\n                \"SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models \"\n                \"(e.g. GQA, MHA). MLA models should not set this flag.\"\n            )\n        self.kv_manager = self._init_kv_manager()\n        if self.enable_staging:\n            self.transfer_queue._init_staging_handler(self.kv_manager)\n\n        if (\n            self.scheduler.tp_worker.is_hybrid_swa\n            and not self._uses_swa_tail_prealloc()\n        ):\n            # Fallback for SWA allocators that still allocate the SWA pool at\n            # full prompt length.\n            self.max_total_num_tokens = min(\n                self.max_total_num_tokens,\n                self.scheduler.tp_worker.model_runner.swa_max_total_num_tokens,\n            )\n","sourceCodeStart":371,"sourceCodeEnd":407,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/disaggregation/decode.py#L371-L407","documentation":"DecodePreallocQueue.__init__ reads SGLANG_DISAGG_STAGING_BUFFER and hard-rejects it for MLA backends. The staging-buffer path was built for non-MLA (GQA/MHA) KV layouts; MLA's compressed latent does not fit the staging buffer scheme, so the flag is treated as a user error rather than silently ignored.","triggerScenarios":"Launching a decode server with SGLANG_DISAGG_STAGING_BUFFER=1 (or true) while the model resolves to an MLA backend (DeepSeek V2/V3, etc.), i.e. self.is_mla_backend is True.","commonSituations":"Copying staging flags from a Qwen/Llama PD deployment to a DeepSeek deployment; experimenting with the staging optimization without reading its model-class restriction.","solutions":["Unset SGLANG_DISAGG_STAGING_BUFFER (or set to 0) for MLA models","For MLA models, use the regular (non-staging) PD transfer path","If you need staging-like behavior for MLA, track upstream support rather than forcing the flag"],"exampleFix":"# before\nexport SGLANG_DISAGG_STAGING_BUFFER=1  # with DeepSeek (MLA)\n# after\nunset SGLANG_DISAGG_STAGING_BUFFER","handlingStrategy":"validation","validationCode":"if envs.SGLANG_DISAGG_STAGING_BUFFER.get():\n    assert not is_mla_model(model_path), 'staging buffer unsupported for MLA'","typeGuard":"def staging_supported(model_backend: str) -> bool:\n    return model_backend not in ('mla', 'hybrid_mla')","tryCatchPattern":null,"preventionTips":["Keep per-model launch templates so MLA configs never carry staging flags"],"tags":["disaggregation","staging-buffer","mla","env-var","unsupported-feature"],"backgroundTag":"unsupported-hardware-feature-combo","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}