{"record":{"id":"07b29386803dd0e6","repo":"sgl-project/sglang","slug":"linear-attn-decode-backend-flashinfer-on-sm100","errorCode":null,"errorMessage":"--linear-attn-decode-backend flashinfer on SM100+ requires --mamba-ssm-dtype bfloat16, got {cfg.mamba_ssm_dtype!r}","messagePattern":"--linear-attn-decode-backend flashinfer on SM100\\+ requires --mamba-ssm-dtype bfloat16, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":6882,"sourceCode":"                    \"--linear-attn-prefill-backend flashkda (decode stays on triton).\"\n                )\n            self._declare(\n                \"_handle_linear_attn_backend\",\n                linear_attn_decode_backend=\"triton\",\n            )\n            decode = \"triton\"\n            logger.info(\n                \"FlashKDA is prefill-only; using triton for KDA decode \"\n                \"(FlashKDA stays on prefill).\"\n            )\n\n        if (\n            decode == \"flashinfer\"\n            and cfg.mamba_ssm_dtype != \"bfloat16\"\n            and is_cuda()\n            and torch.cuda.get_device_capability()[0] >= 10\n        ):\n            raise ValueError(\n                \"--linear-attn-decode-backend flashinfer on SM100+ requires \"\n                \"--mamba-ssm-dtype bfloat16, \"\n                f\"got {cfg.mamba_ssm_dtype!r}\"\n            )\n\n        verify = cfg.linear_attn_verify_backend\n        if verify is None and decode == \"flashinfer\":\n            verify = \"flashinfer\"\n        if (\n            verify == \"flashinfer\"\n            and cfg.mamba_ssm_dtype != \"bfloat16\"\n            and is_cuda()\n            and torch.cuda.get_device_capability()[0] >= 10\n        ):\n            raise ValueError(\n                \"--linear-attn-verify-backend flashinfer on SM100+ requires \"\n                \"--mamba-ssm-dtype bfloat16, \"\n                f\"got {cfg.mamba_ssm_dtype!r}\"","sourceCodeStart":6864,"sourceCodeEnd":6900,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L6864-L6900","documentation":"The FlashInfer linear-attention decode kernel on SM100+ (Blackwell) GPUs is only implemented for bfloat16 SSM state. If --linear-attn-decode-backend flashinfer is combined with any other --mamba-ssm-dtype on a CUDA device with compute capability >= 10, ServerArgs resolution rejects it.","triggerScenarios":"Launching with --linear-attn-decode-backend flashinfer and --mamba-ssm-dtype not bfloat16 (e.g. float16, float32) on a Blackwell GPU (torch.cuda.get_device_capability()[0] >= 10). Non-CUDA platforms or SM < 10 do not trigger this check.","commonSituations":"Reusing a float16 SSM config (tuned for stochastic rounding or Hopper) on a B200 machine while switching the decode backend to flashinfer; defaults drifting after a version upgrade.","solutions":["Add --mamba-ssm-dtype bfloat16 to the launch command","Switch to a decode backend without the bf16 requirement (e.g. triton) if you must keep the current dtype"],"exampleFix":"# before (on SM100+)\n--linear-attn-decode-backend flashinfer --mamba-ssm-dtype float16\n# after\n--linear-attn-decode-backend flashinfer --mamba-ssm-dtype bfloat16","handlingStrategy":"validation","validationCode":"import torch\nif args.linear_attn_decode_backend == \"flashinfer\" and torch.cuda.is_available():\n    if torch.cuda.get_device_capability()[0] >= 10:\n        assert args.mamba_ssm_dtype == \"bfloat16\", \"flashinfer decode on SM100+ requires bf16 SSM dtype\"","typeGuard":"null","tryCatchPattern":null,"preventionTips":["Query compute capability and derive dtype/backend constraints programmatically in your launcher","Track per-architecture dtype requirements when porting configs between Hopper and Blackwell"],"tags":["sglang","linear-attention","flashinfer","sm100","bfloat16","dtype"],"backgroundTag":"incompatible-dtype-configuration","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}