sgl-project/sglang · error · NotImplementedError

PtxKDAKernel does not support target_verify

Error message

PtxKDAKernel does not support target_verify

What it means

PtxKDAKernel is prefill-only and does not implement the speculative-decoding target_verify path; calling it raises NotImplementedError.

Source

Thrown at python/sglang/srt/layers/attention/linear/kernels/kda_ptx.py:82

        self._staging = {}

    def _ensure_loaded(self):
        if self._fwd is None:
            from sglang.kernels.ops.attention.linear.kda_ptx_prefill import (
                chunk_kda_fwd,
                load_ext,
            )

            logger.info("Building the PTX KDA prefill extension (first use, ~1-2 min)")
            load_ext()
            self._fwd = chunk_kda_fwd
            logger.info("Using PTX KDA chunked prefill (GB300 / sm_103a)")

    def decode(self, *args, **kwargs):
        raise NotImplementedError("PtxKDAKernel is prefill-only")

    def target_verify(self, *args, **kwargs):
        raise NotImplementedError("PtxKDAKernel does not support target_verify")

    def _flat_param(self, t: Optional[torch.Tensor]) -> Optional[torch.Tensor]:
        if t is None:
            return None
        key = (t.data_ptr(), t.dtype, tuple(t.shape))
        flat = self._param_flat.get(key)
        if flat is None:
            flat = t.detach().reshape(-1).float().contiguous()
            self._param_flat[key] = flat
        return flat

    def _get_staging(self, bucket, num_heads, head_k_dim, head_v_dim, dev):
        key = (bucket, num_heads, head_k_dim, head_v_dim, dev.index)
        st = self._staging.get(key)
        if st is None:
            st = {
                "q": torch.zeros(
                    1, bucket, num_heads, head_k_dim, dtype=torch.bfloat16, device=dev

View on GitHub (pinned to 0132848349)

Solutions

  1. Switch the verify path to a backend that supports target_verify (e.g. flashinfer)
  2. Disable speculative decoding
  3. Configure per-phase backends so verify avoids the PTX kernel

Example fix

# before
--speculative-algorithm EAGLE  # with PTX KDA backend
# after
--speculative-algorithm NONE   # or use flashinfer for verify
Defensive patterns

Strategy: validation

Validate before calling

if server_args.speculative_algorithm and isinstance(kernel, PtxKDAKernel):
    raise SystemExit('PTX KDA kernel does not support target_verify; disable spec or use flashinfer')

Type guard

def supports_target_verify(kernel) -> bool:
    return type(kernel).target_verify.__code__ is not PtxKDAKernel.target_verify.__code__

Prevention

When it happens

Trigger: Enabling speculative decoding while the PTX KDA kernel is selected, causing target_verify to dispatch to it.

Common situations: Spec decoding (EAGLE) enabled on a KDA model on GB300 hardware with the PTX backend active for verify.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/27b6060b52f8840c. Report an issue: GitHub.