sgl-project/sglang · error · ValueError

FlashInfer GDN prefill is not supported with --enable-determ

Error message

FlashInfer GDN prefill is not supported with --enable-deterministic-inference. Use --linear-attn-prefill-backend triton.

What it means

The GDN (gated deltanet) linear attention backend validates at construction that, when deterministic inference is enabled (--enable-deterministic-inference), the prefill backend is not FlashInfer, because FlashInfer GDN prefill kernels are non-deterministic. _validate_gdn_linear_attn_backends raises this ValueError telling you to use the triton prefill backend instead.

Source

Thrown at python/sglang/srt/layers/attention/linear/gdn_backend.py:118

        return None

    from sglang.srt.layers.attention.linear.kernels.gdn_flashinfer import (
        is_flashinfer_gdn_prefill_available,
    )

    if not is_flashinfer_gdn_prefill_available():
        return None

    rank0_log(f"Defaulting SM{sm_major}0 GDN prefill backend to FlashInfer.")
    return "flashinfer"


def _validate_gdn_linear_attn_backends(backends: LinearAttnBackends) -> None:
    if (
        get_exec().deterministic.enable_deterministic_inference
        and backends.prefill.is_flashinfer()
    ):
        raise ValueError(
            "FlashInfer GDN prefill is not supported with "
            "--enable-deterministic-inference. Use "
            "--linear-attn-prefill-backend triton."
        )


class GDNKernelDispatcher:
    """Dispatches GDN kernel calls to the appropriate backend per mode."""

    def __init__(
        self,
        decode_backend: LinearAttnKernelBackend,
        prefill_backend: LinearAttnKernelBackend,
        verify_backend: Optional[LinearAttnKernelBackend] = None,
    ):
        triton_kernel = TritonGDNKernel()
        self.tree_verify_kernel = triton_kernel

View on GitHub (pinned to 0132848349)

Solutions

1. Pass --linear-attn-prefill-backend triton when using --enable-deterministic-inference
2. If determinism is not required, drop --enable-deterministic-inference to keep FlashInfer prefill
3. Check server startup scripts/defaults that may explicitly set flashinfer prefill and remove the override

Example fix

# before
python -m sglang.launch_server --model Qwen3-Next-80B --enable-deterministic-inference --linear-attn-prefill-backend flashinfer
# after
python -m sglang.launch_server --model Qwen3-Next-80B --enable-deterministic-inference --linear-attn-prefill-backend triton

from sglang.srt.environ import get_exec

def validate_gdn_args(args):
    if get_exec().deterministic.enable_deterministic_inference:
        if args.linear_attn_prefill_backend == 'flashinfer':
            raise SystemExit('use --linear-attn-prefill-backend triton')
        args.linear_attn_prefill_backend = 'triton'

try:
    backend = GDNStrategy(backends)
except ValueError as e:
    if 'deterministic' in str(e):
        backends.prefill = LinearAttnPrefillBackend.TRITON
        backend = GDNStrategy(backends)
    else:
        raise

When it happens

Trigger: Constructing the GDN backend with get_exec().deterministic.enable_deterministic_inference true and LinearAttnBackends.prefill set to flashinfer (explicitly via --linear-attn-prefill-backend flashinfer or as an inferred default), for models like Qwen3-Next.

Common situations: User enables deterministic inference for reproducible serving of a hybrid GDN model while leaving/forcing FlashInfer as the linear-attn prefill backend; defaults change across versions making flashinfer the default prefill backend.

Related errors

• combined_history=True requires direction=0 (bidi)
• Expected hybrid GDN or NemotronH models, but got unknown mod
• --linear-attn-decode-backend flashinfer on SM100+ requires -
• --linear-attn-verify-backend flashinfer on SM100+ requires -
• --linear-attn-prefill-backend flashinfer on SM100+ requires

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9f240fa34978a7fd. Report an issue: GitHub.