sgl-project/sglang · error · NotImplementedError

NvidiaKDAKernel does not support target_verify

Error message

NvidiaKDAKernel does not support target_verify

What it means

NvidiaKDAKernel supports only chunked prefill; target_verify for speculative decoding is not implemented and raises NotImplementedError immediately.

Source

Thrown at python/sglang/srt/layers/attention/linear/kernels/kda_nvidia.py:116

        # (batch size, bucket, shape, device) -> staging dict.
        self._staging = {}

    def _ensure_loaded(self):
        if self._fwd is None:
            from sglang.kernels.ops.attention.fla.l2norm import l2norm_fwd
            from sglang.kernels.ops.attention.linear.kda_nvidia_prefill import (
                chunk_kda_fwd,
            )

            self._fwd = chunk_kda_fwd
            self._l2norm = l2norm_fwd
            logger.info("Using NVIDIA chunked KDA prefill (Blackwell)")

    def decode(self, *args, **kwargs):
        raise NotImplementedError("NvidiaKDAKernel is prefill-only")

    def target_verify(self, *args, **kwargs):
        raise NotImplementedError("NvidiaKDAKernel does not support target_verify")

    def _triton_extend(
        self,
        q,
        k,
        v,
        g,
        beta,
        ssm_states,
        cache_indices,
        query_start_loc,
        A_log,
        dt_bias,
        lower_bound,
        return_intermediate_states,
        kwargs,
    ):
        return self._triton.extend(

View on GitHub (pinned to 0132848349)

Solutions

  1. Use a backend with target_verify support (flashinfer) for spec decoding
  2. Disable speculative decoding
  3. Split backends per phase so verify is not routed to the NVIDIA kernel

Example fix

# before
--linear-attn-backend nvidia --speculative-algorithm EAGLE
# after
--speculative-algorithm NONE   # or route verify to flashinfer
Defensive patterns

Strategy: validation

Validate before calling

if server_args.speculative_algorithm and kda_backend_for_verify in ('nvidia',):
    raise SystemExit('NVIDIA KDA kernel does not support target_verify; use flashinfer or disable spec')

Type guard

def supports_target_verify(kernel) -> bool:
    return type(kernel).target_verify.__code__ is not NvidiaKDAKernel.target_verify.__code__

Prevention

When it happens

Trigger: Enabling EAGLE speculative decoding while the NVIDIA chunked KDA kernel is the active verify backend; target_verify is dispatched to it.

Common situations: Spec decoding enabled on a KDA model whose KDA backend resolves to the NVIDIA prefill kernel for all phases.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c789587d8147d1df. Report an issue: GitHub.