{"record":{"id":"956e9d61ef0ce663","repo":"sgl-project/sglang","slug":"sglang-disagg-staging-buffer-is-designed-for-non-m-956e9d","errorCode":null,"errorMessage":"SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models (e.g. GQA, MHA). MLA models should not set this flag.","messagePattern":"SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models \\(e\\.g\\. GQA, MHA\\)\\. MLA models should not set this flag\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/disaggregation/prefill.py","lineNumber":161,"sourceCode":"        self.is_mla_backend = is_mla_backend(token_to_kv_pool)\n        self.metadata_buffers = metadata_buffers\n        self.req_to_metadata_buffer_idx_allocator = req_to_metadata_buffer_idx_allocator\n        self.tp_rank = tp_rank\n        self.tp_size = tp_size\n        self.pp_rank = pp_rank\n        self.pp_size = pp_size\n        self.gpu_id = gpu_id\n        self.bootstrap_port = bootstrap_port\n        self.queue: List[Req] = []\n        self.gloo_group = gloo_group\n        self.scheduler = scheduler\n        self.max_total_num_tokens = (\n            self.scheduler.tp_worker.model_runner.effective_max_total_num_tokens\n        )\n        self.transfer_backend = transfer_backend\n        if envs.SGLANG_DISAGG_STAGING_BUFFER.get():\n            if self.is_mla_backend:\n                raise RuntimeError(\n                    \"SGLANG_DISAGG_STAGING_BUFFER is designed for non-MLA models \"\n                    \"(e.g. GQA, MHA). MLA models should not set this flag.\"\n                )\n            page_size = self.scheduler.token_to_kv_pool_allocator.page_size\n            # Same source as send_kv_chunk's staging grid below, so validation\n            # and the grid cannot disagree after a post-publish override.\n            chunked_prefill_size = get_schedule().chunked_prefill_size\n            cps = chunked_prefill_size or 8192\n            # Staging slices each send into a fixed page-aligned grid, so an\n            # unbounded (-1) or non-page-aligned chunk size has no valid grid.\n            if cps <= 0 or cps % page_size != 0:\n                raise RuntimeError(\n                    f\"SGLANG_DISAGG_STAGING_BUFFER requires a positive \"\n                    f\"chunked_prefill_size that is a multiple of page_size \"\n                    f\"({page_size}); got {chunked_prefill_size}.\"\n                )\n            if self.pp_size > 1 and self.transfer_backend != TransferBackend.MOONCAKE:\n                raise RuntimeError(","sourceCodeStart":143,"sourceCodeEnd":179,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/disaggregation/prefill.py#L143-L179","documentation":"Configuration guard in DisaggregationPrefillController.__init__: the env flag SGLANG_DISAGG_STAGING_BUFFER enables a page-aligned staging grid for KV sends that is only implemented for non-MLA attention backends (GQA/MHA). MLA models have a different KV layout so the flag is rejected at startup.","triggerScenarios":"Setting SGLANG_DISAGG_STAGING_BUFFER=1 (or true) while launching a disaggregated prefill server with an MLA model (e.g. DeepSeek-V2/V3 style attention), making scheduler.tp_worker's model runner report is_mla_backend true.","commonSituations":"Enabling the staging-buffer optimization to reduce NIXL/Mooncake transfer overhead and then running DeepSeek or another MLA-architecture model; or switching models in a launch script that already exports the env var.","solutions":["Unset SGLANG_DISAGG_STAGING_BUFFER for MLA models (remove it from the launch env / .env).","If staging sends are required, use a GQA/MHA model where the flag is supported.","Track upstream SGLang for MLA staging-buffer support instead of forcing the flag."],"exampleFix":"# before\nexport SGLANG_DISAGG_STAGING_BUFFER=1\npython -m sglang.launch_server --model deepseek-ai/DeepSeek-V3 --disaggregation-prefill ...\n# after (MLA: do not set the flag)\nunset SGLANG_DISAGG_STAGING_BUFFER\npython -m sglang.launch_server --model deepseek-ai/DeepSeek-V3 --disaggregation-prefill ...","handlingStrategy":"validation","validationCode":"# before launching\nimport os, subprocess\nis_mla = model_uses_mla(MODEL_PATH)  # check config attention architecture\nif os.environ.get('SGLANG_DISAGG_STAGING_BUFFER') and is_mla:\n    del os.environ['SGLANG_DISAGG_STAGING_BUFFER']","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep per-model launch scripts so MLA models never inherit non-MLA env flags.","Audit exported SGLANG_* vars when switching model architectures."],"tags":["sglang","disaggregation","env-var","mla","config-validation"],"backgroundTag":"unsupported-feature-configuration","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}