{"record":{"id":"b89ec25e68450cca","repo":"sgl-project/sglang","slug":"qprep-bf16-fp8-sm90-requires-an-sm90-hopper-gpu","errorCode":null,"errorMessage":"qprep_bf16_fp8_sm90 requires an SM90 (Hopper) GPU","messagePattern":"qprep_bf16_fp8_sm90 requires an SM90 \\(Hopper\\) GPU","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/qprep_bf16_fp8_sm90.py","lineNumber":31,"sourceCode":"from typing import TYPE_CHECKING\n\nimport torch\n\nfrom sglang.kernels.jit.utils import cache_once, load_jit\nfrom sglang.kernels.kernel_api_logging import debug_kernel_api\n\nif TYPE_CHECKING:\n    from tvm_ffi.module import Module\n\n\nN_LORA = 512  # kv_lora_rank (nope output dim)\nROPE_DIM = 64  # qk_rope_head_dim\n\n\n@cache_once\ndef _jit_qprep_bf16_fp8_module() -> Module:\n    if torch.cuda.get_device_capability()[0] != 9:\n        raise RuntimeError(\"qprep_bf16_fp8_sm90 requires an SM90 (Hopper) GPU\")\n    return load_jit(\n        \"qprep_bf16_fp8_sm90\",\n        cuda_files=[\"qprep_bf16_fp8_sm90/entry.cuh\"],\n        cuda_wrappers=[(\"dispatch\", \"qprep_bf16_fp8_dispatch\")],\n        # Same minimal flag set as the sparse_mla_q8kv8_prefill_sm90 JIT\n        # build (per-flag ablation there showed the rest are no-ops).\n        extra_cuda_cflags=[\n            \"-O3\",\n            \"-DNDEBUG\",\n            \"-DCUTE_USE_PACKED_TUPLE=1\",\n            \"-DCUTLASS_ENABLE_TENSOR_CORE_MMA=1\",\n            \"--use_fast_math\",\n        ],\n        extra_dependencies=[\"cutlass\"],\n    )\n\n\n# torch._C._cuda_getCurrentRawStream returns the cudaStream_t pointer expected","sourceCodeStart":13,"sourceCodeEnd":49,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/qprep_bf16_fp8_sm90.py#L13-L49","documentation":"qprep_bf16_fp8_sm90's JIT module is gated on the GPU architecture: its CUDA source uses SM90 (Hopper) features, so _jit_qprep_bf16_fp8_module raises RuntimeError when torch.cuda.get_device_capability()[0] != 9. This triggers lazily on the first call via q8kv8_qprep_fwd.","triggerScenarios":"Calling q8kv8_qprep_fwd on any non-Hopper GPU (A100 SM80, Blackwell SM100, or older) which invokes the @cache_once JIT loader.","commonSituations":"Running the q8kv8 sparse prefill pipeline on A100/Ada/Blackwell machines; CI runners without H100; version upgrades enabling this qprep path by default on heterogeneous clusters.","solutions":["Run on an H100/H200 (SM90) GPU","Disable the bf16->fp8 qprep path / select a different quantized prefill kernel on non-SM90 hardware","Gate backend selection on the device capability before the first forward"],"exampleFix":"# before\nout = q8kv8_qprep_fwd(q, ...)  # on A100 -> RuntimeError\n# after\nif torch.cuda.get_device_capability()[0] == 9:\n    out = q8kv8_qprep_fwd(q, ...)\nelse:\n    out = fallback_qprep(q, ...)","handlingStrategy":"fallback","validationCode":"if torch.cuda.get_device_capability()[0] != 9:\n    qprep = fallback_qprep  # non-SM90 path\nelse:\n    from ... import q8kv8_qprep_fwd as qprep","typeGuard":null,"tryCatchPattern":"except RuntimeError as e: if 'SM90' in str(e): switch to non-quantized/fallback prefill","preventionTips":["Gate SM90-only JIT kernels behind capability checks at backend selection","Pin serving hardware to H100/H200 for the q8kv8 pipeline"],"tags":["cuda","sm90","hopper","jit-kernel","hardware-unsupported"],"backgroundTag":"unsupported-gpu-compute-capability","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}