{"record":{"id":"d80b737d9b5546b2","repo":"sgl-project/sglang","slug":"trtllm-mha-backend-for-decode-is-only-supported-on","errorCode":null,"errorMessage":"TRTLLM MHA backend for decode is only supported on Hopper (SM90), Blackwell (SM100) and (SM120) GPUs. Please use a different decode backend.","messagePattern":"TRTLLM MHA backend for decode is only supported on Hopper \\(SM90\\), Blackwell \\(SM100\\) and \\(SM120\\) GPUs\\. Please use a different decode backend\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":6580,"sourceCode":"                prefill_backend == \"trtllm_mha\"\n                and is_sm120_supported()\n                and (\n                    cfg.kv_cache_dtype == \"fp8_e4m3\"\n                    or (\n                        envs.SGLANG_SKIP_SOFTMAX_PREFILL_THRESHOLD_SCALE_FACTOR.get()\n                        or 0.0\n                    )\n                    > 0\n                )\n            ):\n                raise ValueError(\n                    \"TRTLLM FMHAv2 prefill on SM120 does not support \"\n                    \"fp8_e4m3 KV cache or skip-softmax.\"\n                )\n            if decode_backend == \"trtllm_mha\" and not (\n                is_sm90_supported() or is_sm100_supported() or is_sm120_supported()\n            ):\n                raise ValueError(\n                    \"TRTLLM MHA backend for decode is only supported on Hopper (SM90), Blackwell (SM100) and (SM120) GPUs. Please use a different decode backend.\"\n                )\n            if (\n                prefill_backend == \"trtllm_mha\"\n                and not is_sm100_supported()\n                and (cfg.enable_prefill_context_parallel or cfg.attn_cp_size > 1)\n            ):\n                raise ValueError(\n                    \"Prefill context parallelism with the TRTLLM MHA prefill backend \"\n                    \"requires SM100 (trtllm-gen context kernel): the SM90/SM120 \"\n                    \"fmha_v2 prefill path does not implement CP shard masking.\"\n                )\n\n        run_post_process_pass(self, _attention_backend_fa3_fp8_fallback)\n\n        run_post_process_pass(self, _fa4_page_constraint)\n\n        # AMD platforms backends","sourceCodeStart":6562,"sourceCodeEnd":6598,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L6562-L6598","documentation":"The trtllm_mha decode (context/gen) kernels only exist for Hopper (SM90), Blackwell (SM100), and SM120. Resolution-time checks reject selecting trtllm_mha as the decode backend on any other CUDA architecture.","triggerScenarios":"Passing --decode-attention-backend trtllm_mha (or a unified --attention-backend trtllm_mha) on a GPU where is_sm90_supported(), is_sm100_supported(), and is_sm120_supported() are all false.","commonSituations":"Deploying configs written for H100/B200 onto A100 or RTX Ada nodes; backend profiles that silently resolve decode to trtllm_mha on unsupported hardware.","solutions":["Change decode backend to flashinfer/triton (or another backend supported on your GPU)","Confirm the architecture first: python -c \"import torch;print(torch.cuda.get_device_capability())\"","Move the workload to SM90/SM100/SM120 hardware if trtllm_mha decode is required"],"exampleFix":"# before\npython -m sglang.launch_server --model M --decode-attention-backend trtllm_mha\n# after\npython -m sglang.launch_server --model M --decode-attention-backend flashinfer","handlingStrategy":"validation","validationCode":"import torch\ncap = torch.cuda.get_device_capability(0)\nsm = cap[0] * 10 + cap[1]\nif args.decode_attention_backend == \"trtllm_mha\" and sm not in (90, 100, 120):\n    args.decode_attention_backend = \"flashinfer\"","typeGuard":null,"tryCatchPattern":"try:\n    ServerArgs(**kwargs)\nexcept ValueError as e:\n    if \"TRTLLM MHA backend for decode\" in str(e):\n        kwargs[\"decode_attention_backend\"] = \"flashinfer\"\n        ServerArgs(**kwargs)\n    else:\n        raise","preventionTips":["Derive backend selection from detected compute capability in orchestration scripts","Pin backend choices per hardware tier rather than copying configs across clusters","Smoke-test ServerArgs construction in CI on each GPU tier"],"tags":["sglang","gpu-architecture","attention-backend","trtllm","decode"],"backgroundTag":"gpu-architecture-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}