{"record":{"id":"b3cfbca4a94b25c0","repo":"sgl-project/sglang","slug":"trtllm-mha-backend-for-prefill-requires-hopper-sm","errorCode":null,"errorMessage":"TRTLLM MHA backend for prefill requires Hopper (SM90), Blackwell (SM100), or SM120 GPUs. Please use a different prefill backend.","messagePattern":"TRTLLM MHA backend for prefill requires Hopper \\(SM90\\), Blackwell \\(SM100\\), or SM120 GPUs\\. Please use a different prefill backend\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":6557,"sourceCode":"        # resolution pipeline (arg_groups/overrides.py:\n        # _mla_kv_cache_dtype_checks), invoked here at their legacy slot.\n        from sglang.srt.arg_groups.overrides import _mla_kv_cache_dtype_checks\n\n        run_post_process_pass(self, _mla_kv_cache_dtype_checks)\n\n        # The CuteDSL MLA validation + prefill fill moved to the resolution\n        # pipeline (arg_groups/overrides.py: _cutedsl_prefill_backend_fill),\n        # invoked here at its legacy slot.\n        from sglang.srt.arg_groups.overrides import _cutedsl_prefill_backend_fill\n\n        run_post_process_pass(self, _cutedsl_prefill_backend_fill)\n\n        prefill_backend, decode_backend = self._resolved_attention_backends()\n        if \"trtllm_mha\" in (prefill_backend, decode_backend):\n            if prefill_backend == \"trtllm_mha\" and not (\n                is_sm90_supported() or is_sm100_supported() or is_sm120_supported()\n            ):\n                raise ValueError(\n                    \"TRTLLM MHA backend for prefill requires Hopper (SM90), Blackwell (SM100), or SM120 GPUs. \"\n                    \"Please use a different prefill backend.\"\n                )\n            if (\n                prefill_backend == \"trtllm_mha\"\n                and is_sm120_supported()\n                and (\n                    cfg.kv_cache_dtype == \"fp8_e4m3\"\n                    or (\n                        envs.SGLANG_SKIP_SOFTMAX_PREFILL_THRESHOLD_SCALE_FACTOR.get()\n                        or 0.0\n                    )\n                    > 0\n                )\n            ):\n                raise ValueError(\n                    \"TRTLLM FMHAv2 prefill on SM120 does not support \"\n                    \"fp8_e4m3 KV cache or skip-softmax.\"","sourceCodeStart":6539,"sourceCodeEnd":6575,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L6539-L6575","documentation":"The trtllm_mha prefill backend's kernels are only compiled for Hopper (SM90), Blackwell (SM100), and SM120 GPUs. During server-arg resolution SGLang probes the current GPU's compute capability and refuses to select trtllm_mha for prefill on older architectures.","triggerScenarios":"Passing --prefill-attention-backend trtllm_mha (or --attention-backend trtllm_mha) on a GPU where is_sm90_supported(), is_sm100_supported(), and is_sm120_supported() all return false (Ampere/Ada/older).","commonSituations":"Reusing an H100/B200-tuned launch script on an A100 or L40S machine; CI runners with older GPUs; a default backend profile silently resolving prefill to trtllm_mha on unsupported hardware.","solutions":["Switch prefill to a portable backend: --prefill-attention-backend fa3 (Hopper) or triton/flashinfer","Verify the GPU first: nvidia-smi or python -c \"import torch;print(torch.cuda.get_device_capability())\"","Run on SM90/SM100/SM120 hardware if trtllm_mha prefill is required"],"exampleFix":"# before\npython -m sglang.launch_server --model M --prefill-attention-backend trtllm_mha\n# after\npython -m sglang.launch_server --model M --prefill-attention-backend triton --decode-attention-backend trtllm_mha","handlingStrategy":"validation","validationCode":"import torch\ncap = torch.cuda.get_device_capability(0)\nsm = cap[0] * 10 + cap[1]\nif args.prefill_attention_backend == \"trtllm_mha\" and sm not in (90, 100, 120):\n    args.prefill_attention_backend = \"triton\"  # or fa3 on SM90","typeGuard":null,"tryCatchPattern":"try:\n    ServerArgs(**kwargs)\nexcept ValueError as e:\n    if \"TRTLLM MHA backend for prefill\" in str(e):\n        kwargs[\"prefill_attention_backend\"] = \"triton\"\n        ServerArgs(**kwargs)\n    else:\n        raise","preventionTips":["Query torch.cuda.get_device_capability() in launch scripts and select backends programmatically","Gate trtllm_mha behind an SM90/SM100/120 check in deployment templates","Run a dry arg-resolution pass before committing GPU jobs"],"tags":["sglang","gpu-architecture","attention-backend","trtllm","sm90"],"backgroundTag":"gpu-architecture-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}