sgl-project/sglang · error · NotImplementedError

FlashInferKDAKernel has no prefill kernel; keep prefill on T

Error message

FlashInferKDAKernel has no prefill kernel; keep prefill on Triton / CuTe DSL.

What it means

FlashInfer provides only recurrent (decode/verify) KDA kernels, not chunked prefill. The dispatcher is expected to keep prefill on Triton or CuTe DSL; calling extend() directly on FlashInferKDAKernel raises NotImplementedError.

Source

Thrown at python/sglang/srt/layers/attention/linear/kernels/kda_flashinfer.py:335

            A_log=A_log_fi,
            dt_bias=dt_bias_fi,
            scale=None,
            initial_state=state_pool,
            output_final_state=False,
            use_qk_l2norm_in_kernel=True,
            use_gate_in_kernel=True,
            lower_bound=lower_bound,
            cu_seqlens=query_start_loc.to(torch.int32),
            ssm_state_indices=ssm_state_indices,
            num_spec_tokens=num_spec_tokens,
        )

        return output_fi.view(1, seq_len, num_v_heads, head_v_dim)

    # ---- extend (prefill): not provided by FlashInfer ----

    def extend(self, *args, **kwargs):
        raise NotImplementedError(
            "FlashInferKDAKernel has no prefill kernel; keep prefill on Triton / CuTe DSL."
        )

View on GitHub (pinned to 0132848349)

Solutions

Let the KDA dispatcher split phases: prefill on triton/cutedsl, decode on flashinfer
Configure prefill backend separately (triton or cutedsl) when decode backend is flashinfer
Do not call extend() on FlashInferKDAKernel directly

Example fix

# before
kernel = FlashInferKDAKernel(); out = kernel.extend(...)
# after
prefill_kernel = TritonKDAKernel(); out = prefill_kernel.extend(...)

Defensive patterns

Strategy: type-guard

Validate before calling

if phase == 'extend' and isinstance(kernel, FlashInferKDAKernel):
    kernel = prefill_kernel  # triton or cutedsl

Type guard

def kernel_supports_extend(kernel) -> bool:
    return type(kernel).extend is not FlashInferKDAKernel.extend

Prevention

Use the built-in dispatcher's per-phase backend split instead of a single backend
Unit-test each phase route in custom dispatchers

When it happens

Trigger: Explicitly routing prefill/extend to the FlashInfer KDA kernel (e.g. forcing one backend for all phases instead of the split dispatch).

Common situations: Setting --linear-attn-backend flashinfer expecting it to handle all phases; custom dispatcher changes that removed the prefill fallback to triton/cutedsl.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/29d944655010e17c. Report an issue: GitHub.