{"record":{"id":"c2082e7d0338e889","repo":"sgl-project/sglang","slug":"flashinfer-kda-kernel-recurrent-kda-is-not-avail","errorCode":null,"errorMessage":"FlashInfer KDA kernel (recurrent_kda) is not available. Requires SM100 (Blackwell) and a FlashInfer build with KDA support.","messagePattern":"FlashInfer KDA kernel \\(recurrent_kda\\) is not available\\. Requires SM100 \\(Blackwell\\) and a FlashInfer build with KDA support\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/linear/kernels/kda_flashinfer.py","lineNumber":70,"sourceCode":"                logger.info(\"FlashInfer KDA kernel (recurrent_kda) loaded successfully\")\n        except (ImportError, RuntimeError) as e:\n            logger.warning(f\"FlashInfer KDA kernel not available: {e}\")\n            _flashinfer_kda_available = False\n            _flashinfer_recurrent_kda = None\n    return _flashinfer_kda_available, _flashinfer_recurrent_kda\n\n\nclass FlashInferKDAKernel(LinearAttnKernelBase):\n    \"\"\"FlashInfer KDA kernel: SM100 decode + MTP (target_verify), topk=1.\n\n    Prefill (``extend``) is intentionally not implemented -- FlashInfer ships no\n    KDA chunk kernel; the dispatcher keeps prefill on Triton / CuTe DSL.\n    \"\"\"\n\n    def __init__(self):\n        available, self._recurrent_kda = _get_flashinfer_kda_kernel()\n        if not available or self._recurrent_kda is None:\n            raise RuntimeError(\n                \"FlashInfer KDA kernel (recurrent_kda) is not available. \"\n                \"Requires SM100 (Blackwell) and a FlashInfer build with KDA support.\"\n            )\n        # Cache the per-layer constant gate-param prep (A_log/dt_bias reshape+cast),\n        # keyed by tensor identity. Layer params are persistent weights so id() is\n        # stable; this removes the per-call reshape/float/contiguous work.\n        self._gate_cache: dict = {}\n        # Cache the constant per-(row-map, batch, T) verify scatter indices\n        # (ssm_state_indices), which never change across verify calls.\n        self._verify_idx_cache: dict = {}\n        # State pools whose stride layout has been validated against the\n        # recurrent_kda contract (per-layer views are pool-stable, so id() is\n        # a stable key — same lifetime argument as _gate_cache).\n        self._state_contract_ok: set = set()\n        logger.info(\"Using FlashInfer KDA kernel\")\n\n    def _check_state_stride_contract(self, ssm_states: torch.Tensor) -> None:\n        \"\"\"One-time (per pool view) check that ``ssm_states`` matches the","sourceCodeStart":52,"sourceCodeEnd":88,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/linear/kernels/kda_flashinfer.py#L52-L88","documentation":"FlashInferKDAKernel.__init__ probes for the recurrent_kda kernel in the installed FlashInfer package; if the symbol is missing or the GPU is not SM100 (Blackwell), it raises RuntimeError at construction time.","triggerScenarios":"Selecting the flashinfer KDA backend on a pre-Blackwell GPU (Hopper/Ampere) or with a FlashInfer wheel built without KDA support.","commonSituations":"Running KDA models on H100/A100 with --linear-attn-backend flashinfer, or using an older/nightly FlashInfer version that predates recurrent_kda.","solutions":["Upgrade FlashInfer to a build that includes recurrent_kda (recent nightly/release)","Run on a Blackwell (SM100) GPU such as B200","Fall back to triton or cutedsl KDA backend"],"exampleFix":"# before\n--linear-attn-backend flashinfer\n# after (on non-Blackwell hardware)\n--linear-attn-backend triton","handlingStrategy":"fallback","validationCode":"import torch\nsm = torch.cuda.get_device_capability(0)[0]\ntry:\n    from flashinfer import recurrent_kda  # or the actual export site\n    has_kda = True\nexcept (ImportError, AttributeError):\n    has_kda = False\nif not (sm >= 100 and has_kda):\n    linear_attn_backend = 'triton'  # fallback","typeGuard":"null","tryCatchPattern":"try:\n    kernel = FlashInferKDAKernel()\nexcept RuntimeError as e:\n    logger.warning('flashinfer KDA unavailable: %s; falling back to triton', e)\n    kernel = TritonKDAKernel()","preventionTips":["Probe torch.cuda.get_device_capability() and FlashInfer symbols at startup","Keep a fallback KDA backend configured in deployment scripts"],"tags":["sglang","flashinfer","kda","gpu-compatibility","sm100"],"backgroundTag":"missing-optional-dependency","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}