{"record":{"id":"da78bc088e7cb2e2","repo":"sgl-project/sglang","slug":"nvidiakdakernel-is-prefill-only","errorCode":null,"errorMessage":"NvidiaKDAKernel is prefill-only","messagePattern":"NvidiaKDAKernel is prefill-only","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/linear/kernels/kda_nvidia.py","lineNumber":113,"sourceCode":"        # One-shot engagement log per (batch size, bucket) (observability).\n        self._engaged_logged = set()\n        self._unsupported_logged = set()\n        # (batch size, bucket, shape, device) -> staging dict.\n        self._staging = {}\n\n    def _ensure_loaded(self):\n        if self._fwd is None:\n            from sglang.kernels.ops.attention.fla.l2norm import l2norm_fwd\n            from sglang.kernels.ops.attention.linear.kda_nvidia_prefill import (\n                chunk_kda_fwd,\n            )\n\n            self._fwd = chunk_kda_fwd\n            self._l2norm = l2norm_fwd\n            logger.info(\"Using NVIDIA chunked KDA prefill (Blackwell)\")\n\n    def decode(self, *args, **kwargs):\n        raise NotImplementedError(\"NvidiaKDAKernel is prefill-only\")\n\n    def target_verify(self, *args, **kwargs):\n        raise NotImplementedError(\"NvidiaKDAKernel does not support target_verify\")\n\n    def _triton_extend(\n        self,\n        q,\n        k,\n        v,\n        g,\n        beta,\n        ssm_states,\n        cache_indices,\n        query_start_loc,\n        A_log,\n        dt_bias,\n        lower_bound,\n        return_intermediate_states,","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/linear/kernels/kda_nvidia.py#L95-L131","documentation":"NvidiaKDAKernel wraps NVIDIA's chunked KDA prefill kernel (Blackwell); it deliberately does not implement decode and raises NotImplementedError to force decode onto another backend.","triggerScenarios":"Dispatching the decode phase to NvidiaKDAKernel — misconfigured single-backend routing or direct API misuse.","commonSituations":"Setting the NVIDIA chunked backend for all phases; custom dispatcher changes removing the decode fallback.","solutions":["Route decode to a decode-capable KDA kernel (triton/flashinfer/helion)","Keep the NVIDIA kernel for prefill (extend) only","Check per-phase backend configuration"],"exampleFix":"# before\nkernel = NvidiaKDAKernel(); out = kernel.decode(...)\n# after\nout = triton_decode_kernel.decode(...)  # nvidia kernel for extend only","handlingStrategy":"type-guard","validationCode":"if phase == 'decode':\n    assert not isinstance(kernel, NvidiaKDAKernel), 'NVIDIA KDA kernel is prefill-only'","typeGuard":"def kernel_supports_decode(kernel) -> bool:\n    return type(kernel).decode.__code__ is not NvidiaKDAKernel.decode.__code__","tryCatchPattern":null,"preventionTips":["Use per-phase backend routing for the NVIDIA chunked kernel","Add phase capability checks in custom dispatchers"],"tags":["sglang","nvidia","kda","decode","not-implemented"],"backgroundTag":"unsupported-operation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}