{"record":{"id":"0a2ba2999cf0e045","repo":"sgl-project/sglang","slug":"attn-res-fused-tma-requires-sm100-excluding-sm12x","errorCode":null,"errorMessage":"attn_res_fused_tma requires SM100+ excluding SM12x; SM{major}{minor} is unsupported","messagePattern":"attn_res_fused_tma requires SM100\\+ excluding SM12x; SM(.+?)(.+?) is unsupported","errorType":"error_code","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kimi_k3/attn_res.py","lineNumber":40,"sourceCode":"_DIM: int = 7168  # K3 hidden size, template parameter of the TMA kernel\n_MAX_BANK_ROWS: int = 8  # K3 has <= 8 snapshots, upper bound of the nvb dispatch tables\n\n\ndef _make_name(*args):\n    return \"kimi_k3_attn_res_\" + \"_\".join(str(a) for a in args)\n\n\n@cache_once\ndef _jit_fused_tma_module(\n    chunk_rows: int, occupancy: int, consumer_regs: int\n) -> Module:\n    \"\"\"Compile and cache the warp-specialized TMA aggregation kernel (per-row\n    bulk copies into chunk slots; chunk_rows / occupancy / consumer_regs are\n    tuning knobs). The smem ring is frozen at 2 chunk slots and PDL is always\n    on: the kernel targets SM100+ except SM12x.\"\"\"\n    major, minor = torch.cuda.get_device_capability()\n    if major < 10 or major == 12:\n        raise RuntimeError(\n            \"attn_res_fused_tma requires SM100+ excluding SM12x; \"\n            f\"SM{major}{minor} is unsupported\"\n        )\n    args = make_cpp_args(\n        _DIM,\n        _MAX_BANK_ROWS,\n        chunk_rows,\n        occupancy,\n        consumer_regs,\n    )\n    with override_jit_cuda_arch(major, minor, suffix=\"a\"):\n        return load_jit(\n            _make_name(\"fused_tma\"),\n            *args,\n            cuda_files=[\"kimi_k3/attn_res/fused_tma.cuh\"],\n            cuda_wrappers=[\n                (\"run\", f\"AttnResFusedTmaKernel<{args}>::run\"),\n                (\"run_pull_rs\", f\"AttnResFusedTmaKernel<{args}>::run_pull_rs\"),","sourceCodeStart":22,"sourceCodeEnd":58,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kimi_k3/attn_res.py#L22-L58","documentation":"attn_res_fused_tma JIT-compiles a warp-specialized TMA kernel that is written for SM100+ (Blackwell) datacenter GPUs, explicitly excluding SM12x (consumer Blackwell). It queries the device capability and raises on any other architecture so an untested GPU never runs a kernel with unsupported TMA/cluster features.","triggerScenarios":"Calling attn_res_fused_tma (or the internal _attn_res_fused_pull_rs_op / _attn_res_fused_direct_ag_op / _precompile) on a GPU with compute capability < 10.0 or in the 12.x family (Hopper SM90, Ada SM89, RTX 50-series SM120, etc.).","commonSituations":"Running the Kimi K3 attention-residual fused path on H100/A100/RTX 5090 or a CI runner without a Blackwell datacenter GPU; a machine with multiple GPUs where device 0 is not the target architecture.","solutions":["Route to a non-TMA fallback (attn_res_fused_direct_ag / pull_rs non-fused path or the plain attention-residual op) when the device is not SM100+","Run on a B200/B100 (SM100) class GPU if the fused TMA path is required","Gate the call behind a check of torch.cuda.get_device_capability() so unsupported GPUs never reach the JIT compile"],"exampleFix":"# before\nout = attn_res_fused_tma(...)\n# after\nmajor, _ = torch.cuda.get_device_capability()\nif major >= 10 and major != 12:\n    out = attn_res_fused_tma(...)\nelse:\n    out = attn_res_fused_direct_ag(...)  # or non-fused fallback","handlingStrategy":"fallback","validationCode":"major, minor = torch.cuda.get_device_capability()\nuse_fused_tma = major >= 10 and major != 12","typeGuard":null,"tryCatchPattern":"try:\n    out = attn_res_fused_tma(...)\nexcept RuntimeError as e:\n    if 'SM100+' in str(e):\n        out = attn_res_fused_direct_ag(...)  # fallback path\n    else:\n        raise","preventionTips":["Cache the capability check once at model init and select the backend there","Never assume datacenter Blackwell in multi-GPU heterogenous boxes","Pin deployment hardware in config and assert at startup"],"tags":["cuda","tma","sm100","architecture-check","jit"],"backgroundTag":"unsupported-gpu-architecture","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}