{"record":{"id":"c789587d8147d1df","repo":"sgl-project/sglang","slug":"nvidiakdakernel-does-not-support-target-verify","errorCode":null,"errorMessage":"NvidiaKDAKernel does not support target_verify","messagePattern":"NvidiaKDAKernel does not support target_verify","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/linear/kernels/kda_nvidia.py","lineNumber":116,"sourceCode":"        # (batch size, bucket, shape, device) -> staging dict.\n        self._staging = {}\n\n    def _ensure_loaded(self):\n        if self._fwd is None:\n            from sglang.kernels.ops.attention.fla.l2norm import l2norm_fwd\n            from sglang.kernels.ops.attention.linear.kda_nvidia_prefill import (\n                chunk_kda_fwd,\n            )\n\n            self._fwd = chunk_kda_fwd\n            self._l2norm = l2norm_fwd\n            logger.info(\"Using NVIDIA chunked KDA prefill (Blackwell)\")\n\n    def decode(self, *args, **kwargs):\n        raise NotImplementedError(\"NvidiaKDAKernel is prefill-only\")\n\n    def target_verify(self, *args, **kwargs):\n        raise NotImplementedError(\"NvidiaKDAKernel does not support target_verify\")\n\n    def _triton_extend(\n        self,\n        q,\n        k,\n        v,\n        g,\n        beta,\n        ssm_states,\n        cache_indices,\n        query_start_loc,\n        A_log,\n        dt_bias,\n        lower_bound,\n        return_intermediate_states,\n        kwargs,\n    ):\n        return self._triton.extend(","sourceCodeStart":98,"sourceCodeEnd":134,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/linear/kernels/kda_nvidia.py#L98-L134","documentation":"NvidiaKDAKernel supports only chunked prefill; target_verify for speculative decoding is not implemented and raises NotImplementedError immediately.","triggerScenarios":"Enabling EAGLE speculative decoding while the NVIDIA chunked KDA kernel is the active verify backend; target_verify is dispatched to it.","commonSituations":"Spec decoding enabled on a KDA model whose KDA backend resolves to the NVIDIA prefill kernel for all phases.","solutions":["Use a backend with target_verify support (flashinfer) for spec decoding","Disable speculative decoding","Split backends per phase so verify is not routed to the NVIDIA kernel"],"exampleFix":"# before\n--linear-attn-backend nvidia --speculative-algorithm EAGLE\n# after\n--speculative-algorithm NONE   # or route verify to flashinfer","handlingStrategy":"validation","validationCode":"if server_args.speculative_algorithm and kda_backend_for_verify in ('nvidia',):\n    raise SystemExit('NVIDIA KDA kernel does not support target_verify; use flashinfer or disable spec')","typeGuard":"def supports_target_verify(kernel) -> bool:\n    return type(kernel).target_verify.__code__ is not NvidiaKDAKernel.target_verify.__code__","tryCatchPattern":null,"preventionTips":["Cross-check spec decoding flags with KDA backend capability before launch"],"tags":["sglang","nvidia","kda","speculative-decoding","not-implemented"],"backgroundTag":"unsupported-operation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}