sgl-project/sglang · error · RuntimeError

FlashInfer KDA kernel (recurrent_kda) is not available. Requ

Error message

FlashInfer KDA kernel (recurrent_kda) is not available. Requires SM100 (Blackwell) and a FlashInfer build with KDA support.

What it means

FlashInferKDAKernel.__init__ probes for the recurrent_kda kernel in the installed FlashInfer package; if the symbol is missing or the GPU is not SM100 (Blackwell), it raises RuntimeError at construction time.

Source

Thrown at python/sglang/srt/layers/attention/linear/kernels/kda_flashinfer.py:70

                logger.info("FlashInfer KDA kernel (recurrent_kda) loaded successfully")
        except (ImportError, RuntimeError) as e:
            logger.warning(f"FlashInfer KDA kernel not available: {e}")
            _flashinfer_kda_available = False
            _flashinfer_recurrent_kda = None
    return _flashinfer_kda_available, _flashinfer_recurrent_kda


class FlashInferKDAKernel(LinearAttnKernelBase):
    """FlashInfer KDA kernel: SM100 decode + MTP (target_verify), topk=1.

    Prefill (``extend``) is intentionally not implemented -- FlashInfer ships no
    KDA chunk kernel; the dispatcher keeps prefill on Triton / CuTe DSL.
    """

    def __init__(self):
        available, self._recurrent_kda = _get_flashinfer_kda_kernel()
        if not available or self._recurrent_kda is None:
            raise RuntimeError(
                "FlashInfer KDA kernel (recurrent_kda) is not available. "
                "Requires SM100 (Blackwell) and a FlashInfer build with KDA support."
            )
        # Cache the per-layer constant gate-param prep (A_log/dt_bias reshape+cast),
        # keyed by tensor identity. Layer params are persistent weights so id() is
        # stable; this removes the per-call reshape/float/contiguous work.
        self._gate_cache: dict = {}
        # Cache the constant per-(row-map, batch, T) verify scatter indices
        # (ssm_state_indices), which never change across verify calls.
        self._verify_idx_cache: dict = {}
        # State pools whose stride layout has been validated against the
        # recurrent_kda contract (per-layer views are pool-stable, so id() is
        # a stable key — same lifetime argument as _gate_cache).
        self._state_contract_ok: set = set()
        logger.info("Using FlashInfer KDA kernel")

    def _check_state_stride_contract(self, ssm_states: torch.Tensor) -> None:
        """One-time (per pool view) check that ``ssm_states`` matches the

View on GitHub (pinned to 0132848349)

Solutions

  1. Upgrade FlashInfer to a build that includes recurrent_kda (recent nightly/release)
  2. Run on a Blackwell (SM100) GPU such as B200
  3. Fall back to triton or cutedsl KDA backend

Example fix

# before
--linear-attn-backend flashinfer
# after (on non-Blackwell hardware)
--linear-attn-backend triton
Defensive patterns

Strategy: fallback

Validate before calling

import torch
sm = torch.cuda.get_device_capability(0)[0]
try:
    from flashinfer import recurrent_kda  # or the actual export site
    has_kda = True
except (ImportError, AttributeError):
    has_kda = False
if not (sm >= 100 and has_kda):
    linear_attn_backend = 'triton'  # fallback

Type guard

null

Try / catch

try:
    kernel = FlashInferKDAKernel()
except RuntimeError as e:
    logger.warning('flashinfer KDA unavailable: %s; falling back to triton', e)
    kernel = TritonKDAKernel()

Prevention

When it happens

Trigger: Selecting the flashinfer KDA backend on a pre-Blackwell GPU (Hopper/Ampere) or with a FlashInfer wheel built without KDA support.

Common situations: Running KDA models on H100/A100 with --linear-attn-backend flashinfer, or using an older/nightly FlashInfer version that predates recurrent_kda.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c2082e7d0338e889. Report an issue: GitHub.