{"record":{"id":"71e80fc13aee7f40","repo":"sgl-project/sglang","slug":"linear-attn-prefill-backend-flashinfer-on-sm100","errorCode":null,"errorMessage":"--linear-attn-prefill-backend flashinfer on SM100+ requires CUDA 13+, got CUDA {cuda_version or 'unknown'}","messagePattern":"--linear-attn-prefill-backend flashinfer on SM100\\+ requires CUDA 13\\+, got CUDA (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":6914,"sourceCode":"        ):\n            raise ValueError(\n                \"--linear-attn-verify-backend flashinfer on SM100+ requires \"\n                \"--mamba-ssm-dtype bfloat16, \"\n                f\"got {cfg.mamba_ssm_dtype!r}\"\n            )\n\n        # SM100+ FlashInfer GDN prefill requires CUDA 13+ (CuTe DSL kernel)\n        # for correctness and best performance.\n        prefill = cfg.linear_attn_prefill_backend or cfg.linear_attn_backend\n        cuda_version = torch.version.cuda\n        cuda_major = int(cuda_version.split(\".\")[0]) if cuda_version is not None else 0\n        if (\n            prefill == \"flashinfer\"\n            and is_cuda()\n            and torch.cuda.get_device_capability()[0] >= 10\n            and cuda_major < 13\n        ):\n            raise ValueError(\n                \"--linear-attn-prefill-backend flashinfer on SM100+ requires CUDA 13+, \"\n                f\"got CUDA {cuda_version or 'unknown'}\"\n            )\n\n        # ReplaySSM buffered decode guards. Runs on Triton, or Helion for KDA.\n        # cuda-graph is supported (slice 1b: CUDA-graph-safe static\n        # write-cursor buffers). The RADIX prefix cache is now supported (slice\n        # 2b: the decode kernel force-flushes the ring into temporal[slot] on\n        # the radix track boundary `seq_lens % mamba_track_interval == 0`, and\n        # the COW copy-into-slot path resets the ring cursor) -- so the\n        # --disable-radix-cache requirement is dropped.\n        #\n        # Slice 2b only wires the no_buffer mamba scheduler strategy (the\n        # default). The extra_buffer strategy donates the track snapshot via\n        # `donate_mamba_ping_pong_slot` with a separate ping-pong slot swap that\n        # does NOT route through MambaPool.copy_from, so the ReplaySSM ring\n        # cursor of the donated/kept slot would not be reset there. Handling\n        # that donation path is a follow-up; for now require no_buffer.","sourceCodeStart":6896,"sourceCodeEnd":6932,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L6896-L6932","documentation":"SGLang rejects --linear-attn-prefill-backend flashinfer on SM100+ (Blackwell) GPUs when the installed CUDA toolkit major version is below 13. The SM100+ FlashInfer GDN prefill path uses a CuTe DSL kernel that requires CUDA 13+ to build/run. Raised during server-args resolution at startup.","triggerScenarios":"--linear-attn-prefill-backend flashinfer on a GPU with compute capability >= 10 while torch.version.cuda / nvcc reports CUDA 12.x (e.g. standard CUDA 12.4/12.6 containers on B200).","commonSituations":"Running Blackwell hardware inside the common CUDA 12.x NGC/pytorch images; upgrading GPUs without upgrading the CUDA toolkit/container; CI images pinned to CUDA 12.","solutions":["Upgrade to a CUDA 13+ build/container (e.g. pytorch:nightly or ngc image with CUDA 13) and restart","Or switch prefill backend on CUDA 12: --linear-attn-prefill-backend triton","Verify with python -c \"import torch; print(torch.version.cuda, torch.cuda.get_device_capability())\""],"exampleFix":"# before (CUDA 12.x container)\n--linear-attn-prefill-backend flashinfer\n# after\n# option A: run under CUDA 13+ image\n# option B: --linear-attn-prefill-backend triton","handlingStrategy":"validation","validationCode":"import torch\ndef cuda_major():\n    v = torch.version.cuda\n    return int(v.split('.')[0]) if v else 0\ndef ok_flashinfer_prefill():\n    return cuda_major() >= 13 or (not torch.cuda.is_available()) or torch.cuda.get_device_capability()[0] < 10","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pin deployment containers to CUDA 13+ on Blackwell fleets","Assert torch.version.cuda >= 13 in smoke tests before server launch"],"tags":["sglang","flashinfer","cuda-version","blackwell","linear-attention"],"backgroundTag":"cuda-version-too-old","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}