sgl-project/sglang · error · NotImplementedError

allow_neg_eigval=True requires 2*sigmoid(beta), which is not

Error message

allow_neg_eigval=True requires 2*sigmoid(beta), which is not implemented by the fused beta path; pass pre-activated beta with use_beta_sigmoid_in_kernel=False

What it means

The fused beta path computes sigmoid(beta) inside the kernel, which forces beta >= 0 semantics. allow_neg_eigval=True needs the 2*sigmoid(beta) parameterization which that fused path does not implement, so the combination is rejected.

Source

Thrown at python/sglang/kernels/ops/attention/linear/kda_ptx_prefill/__init__.py:119

        table -- pass cu_seqlens_cpu to avoid the D2H sync; chunk_indices is
        accepted and ignored (the kernel derives its own piece table).
      initial_state [N,H,128,128] fp32 or None (zeros).
      return_intermediate_states=True returns dense fp32 chunk-boundary states
        [1, NT, H, 128, 128] at tuple index 10.

    Returns the fla-shaped 12-tuple: (o [B,T,H,128] bf16, final_state
    [N,H,128,128] fp32 or None, then Nones, ..., h, initial_state).
    """
    assert (
        chunk_size == CHUNK
    ), f"kda_prefill supports chunk_size={CHUNK} only, got {chunk_size}"
    if cp_context is not None or disable_recompute:
        raise NotImplementedError(
            "kda_prefill is the inference forward path: cp_context, "
            "and disable_recompute are training-side knobs it does not implement"
        )
    if allow_neg_eigval and use_beta_sigmoid_in_kernel:
        raise NotImplementedError(
            "allow_neg_eigval=True requires 2*sigmoid(beta), which is not "
            "implemented by the fused beta path; pass pre-activated beta with "
            "use_beta_sigmoid_in_kernel=False"
        )
    if state_v_first and initial_state is not None:
        # [V,K]-layout state: pure transpose (K==V==128), exact, ~us/call
        initial_state = initial_state.transpose(-1, -2).contiguous()
    assert (
        q.dim() == 4 and q.shape[-1] == K and v.shape[-1] == K
    ), f"expected [B,T,H,{K}] q/k/v, got q={tuple(q.shape)} v={tuple(v.shape)}"
    B, T, H, _ = q.shape

    cu_cpu = None
    if cu_seqlens is not None or cu_seqlens_cpu is not None:
        assert B == 1, "cu_seqlens requires B == 1 (flattened varlen batch)"
        src = cu_seqlens_cpu if cu_seqlens_cpu is not None else cu_seqlens
        cu_cpu = torch.as_tensor(src, dtype=torch.int32).cpu()
    elif B > 1:

View on GitHub (pinned to 0132848349)

Solutions

Pre-activate beta yourself (beta = 2*sigmoid(beta_raw)) and pass use_beta_sigmoid_in_kernel=False
If you do not need negative eigenvalues, set allow_neg_eigval=False

Example fix

// before
kda_prefill(q, k, v, beta=beta_raw, allow_neg_eigval=True, use_beta_sigmoid_in_kernel=True)
// after
beta = torch.sigmoid(beta_raw) * 2  # pre-activated
kda_prefill(q, k, v, beta=beta, allow_neg_eigval=True, use_beta_sigmoid_in_kernel=False)

Defensive patterns

Strategy: validation

Validate before calling

if allow_neg_eigval:
    beta = torch.sigmoid(beta) * 2
    use_beta_sigmoid_in_kernel = False

Prevention

Decide beta activation ownership (caller vs kernel) once at model init
Check allow_neg_eigval × use_beta_sigmoid_in_kernel compatibility in config validation

When it happens

Trigger: Calling kda_prefill with allow_neg_eigval=True and use_beta_sigmoid_in_kernel=True (the default fused beta activation).

Common situations: Loading a KDA checkpoint trained with negative eigenvalues (allow_neg_eigval=True) while keeping the kernel-side sigmoid default; upgrading configs where beta is now expected pre-activated.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d98687be7afe3d5c. Report an issue: GitHub.