sgl-project/sglang · error · NotImplementedError

NvidiaKDAKernel is prefill-only

Error message

NvidiaKDAKernel is prefill-only

What it means

NvidiaKDAKernel wraps NVIDIA's chunked KDA prefill kernel (Blackwell); it deliberately does not implement decode and raises NotImplementedError to force decode onto another backend.

Source

Thrown at python/sglang/srt/layers/attention/linear/kernels/kda_nvidia.py:113

        # One-shot engagement log per (batch size, bucket) (observability).
        self._engaged_logged = set()
        self._unsupported_logged = set()
        # (batch size, bucket, shape, device) -> staging dict.
        self._staging = {}

    def _ensure_loaded(self):
        if self._fwd is None:
            from sglang.kernels.ops.attention.fla.l2norm import l2norm_fwd
            from sglang.kernels.ops.attention.linear.kda_nvidia_prefill import (
                chunk_kda_fwd,
            )

            self._fwd = chunk_kda_fwd
            self._l2norm = l2norm_fwd
            logger.info("Using NVIDIA chunked KDA prefill (Blackwell)")

    def decode(self, *args, **kwargs):
        raise NotImplementedError("NvidiaKDAKernel is prefill-only")

    def target_verify(self, *args, **kwargs):
        raise NotImplementedError("NvidiaKDAKernel does not support target_verify")

    def _triton_extend(
        self,
        q,
        k,
        v,
        g,
        beta,
        ssm_states,
        cache_indices,
        query_start_loc,
        A_log,
        dt_bias,
        lower_bound,
        return_intermediate_states,

View on GitHub (pinned to 0132848349)

Solutions

1. Route decode to a decode-capable KDA kernel (triton/flashinfer/helion)
2. Keep the NVIDIA kernel for prefill (extend) only
3. Check per-phase backend configuration

Example fix

# before
kernel = NvidiaKDAKernel(); out = kernel.decode(...)
# after
out = triton_decode_kernel.decode(...)  # nvidia kernel for extend only

if phase == 'decode':
    assert not isinstance(kernel, NvidiaKDAKernel), 'NVIDIA KDA kernel is prefill-only'

def kernel_supports_decode(kernel) -> bool:
    return type(kernel).decode.__code__ is not NvidiaKDAKernel.decode.__code__

When it happens

Trigger: Dispatching the decode phase to NvidiaKDAKernel — misconfigured single-backend routing or direct API misuse.

Common situations: Setting the NVIDIA chunked backend for all phases; custom dispatcher changes removing the decode fallback.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• FlashKDAKernel only supports prefill (extend)
• NvidiaKDAKernel does not support target_verify
• PtxKDAKernel is prefill-only
• CuteDSLKDAKernel does not support target_verify
• FlashInferKDAKernel has no prefill kernel; keep prefill on T

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/da78bc088e7cb2e2. Report an issue: GitHub.