{"record":{"id":"da18d3a83a2bf6ee","repo":"sgl-project/sglang","slug":"deterministic-inference-with-absorbed-mla-models-o","errorCode":null,"errorMessage":"Deterministic inference with absorbed-MLA models on the fa4 attention backend requires SM100/SM110: it runs absorbed MLA, whose qv argument flash_attn.cute only implements on those archs.","messagePattern":"Deterministic inference with absorbed-MLA models on the fa4 attention backend requires SM100/SM110: it runs absorbed MLA, whose qv argument flash_attn\\.cute only implements on those archs\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":9238,"sourceCode":"                        \"Glm4MoeLiteForCausalLM\",\n                    ]\n                except Exception:\n                    pass\n\n            # Check attention backend\n            run_post_process_pass(self, _deterministic_attention_backend)\n\n            attention_backend = resolved_view(self).attention_backend\n            if is_deepseek_model:\n                if (\n                    attention_backend\n                    not in RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACKEND\n                ):\n                    raise ValueError(\n                        f\"Currently only {RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACKEND} attention backends are supported for deterministic inference with absorbed-MLA models. But you're using {attention_backend}.\"\n                    )\n                if attention_backend == \"fa4\" and not is_sm100_or_sm110_supported():\n                    raise ValueError(\n                        \"Deterministic inference with absorbed-MLA models on the fa4 \"\n                        \"attention backend requires SM100/SM110: it runs \"\n                        \"absorbed MLA, whose qv argument flash_attn.cute only \"\n                        \"implements on those archs.\"\n                    )\n\n            if attention_backend not in RADIX_SUPPORTED_DETERMINISTIC_ATTENTION_BACKEND:\n                # Currently, only certain backends support radix cache. Support for other backends is in progress\n                self._declare(\n                    \"_handle_deterministic_inference\",\n                    disable_radix_cache=True,\n                )\n                logger.warning(\n                    f\"Currently radix cache is not compatible with {attention_backend} attention backend for deterministic inference. It will be supported in the future.\"\n                )\n\n            # Check TP size\n            if cfg.tp_size > 1:","sourceCodeStart":9220,"sourceCodeEnd":9256,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L9220-L9256","documentation":"Raised when deterministic inference with an absorbed-MLA model selects the fa4 (flash_attn.cute) backend on hardware that is not SM100/SM110 (Blackwell). The absorbed-MLA qv argument in flash_attn.cute is only implemented for those architectures, so older GPUs cannot run this path deterministically.","triggerScenarios":"Running with --attention-backend fa4 plus deterministic inference on a DeepSeek model on Hopper (H100) or older GPUs; the preceding backend check passes (fa4 is in the supported set) but the arch check fails.","commonSituations":"Developing on H100 with configs written for B200/Blackwell; CI runners with different GPU archs than the target cluster; fa4 newly supported for determinism but only on SM100+.","solutions":["Use a different supported attention backend (e.g. fa3) on non-Blackwell hardware","Or run on SM100/SM110 (B200/GB200-class) GPUs","Or disable deterministic inference if not strictly needed"],"exampleFix":"# before (on H100)\n--attention-backend fa4 --rl-on-policy-target ...\n# after\n--attention-backend fa3 --rl-on-policy-target ...","handlingStrategy":"fallback","validationCode":"import torch\nsm = torch.cuda.get_device_capability()\nif args.attention_backend == 'fa4' and deterministic and sm not in ((10,0),(11,0)):\n    args.attention_backend = 'fa3'","typeGuard":"def fa4_ok_for_this_gpu() -> bool:\n    import torch\n    cc = torch.cuda.get_device_capability()\n    return cc[0] == 10 and cc[1] in (0, 1) or cc == (11, 0)","tryCatchPattern":"except ValueError as e:\n    if 'SM100/SM110' in str(e):\n        retry with --attention-backend fa3","preventionTips":["Gate fa4 usage on torch.cuda.get_device_capability() in launch scripts","Keep an H100-compatible backend config variant"],"tags":["sglang","fa4","cuda-arch","deterministic-inference","blackwell"],"backgroundTag":"gpu-architecture-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}