sgl-project/sglang · error · ValueError

q must be provided unless qv is provided with only_qv=True

Error message

q must be provided unless qv is provided with only_qv=True

What it means

Follow-on guard inside the only_qv path: q is None, only_qv=True, but qv is also None — so the wrapper has no way to infer q's shape/dtype/device to synthesize the dummy q tensor the kernel API requires. Exactly one of q or qv must be provided on the QV path.

Source

Thrown at python/sglang/kernels/aot/python/sgl_kernel/flash_attn.py:192

        elif k is not None:
            k_head_size = k.shape[-1]
            k_dtype = k.dtype
            k_device = k.device
        else:
            # Fallback: only_qv kernel ignores K values, so a tiny placeholder works.
            k_head_size = 64
            k_dtype = v_cache.dtype
            k_device = v_cache.device
        k_shape = (*v_cache.shape[:-1], k_head_size)
        # The kernel path for only_qv ignores K values, but backend API still requires k tensor.
        k_cache = torch.empty(k_shape, dtype=k_dtype, device=k_device)
    assert k_cache.stride(-1) == 1, "k_cache must have contiguous last dimension"

    if q is None:
        if not only_qv:
            raise ValueError("q can only be None when only_qv=True")
        if qv is None:
            raise ValueError(
                "q must be provided unless qv is provided with only_qv=True"
            )
        q_shape = (*qv.shape[:-1], k_cache.shape[-1])
        # The kernel path for only_qv ignores q values, but backend API still requires q tensor.
        q = torch.empty(q_shape, dtype=qv.dtype, device=qv.device)

    if softmax_scale is None:
        if only_qv:
            if qv is None:
                raise ValueError("only_qv=True requires qv to be provided")
            softmax_scale = (qv.shape[-1]) ** (-0.5)
        else:
            softmax_scale = (q.shape[-1] + (qv.shape[-1] if qv is not None else 0)) ** (
                -0.5
            )
    if cache_seqlens is not None and isinstance(cache_seqlens, int):
        cache_seqlens = torch.full(
            (q.shape[0],), cache_seqlens, dtype=torch.int32, device=v_cache.device

View on GitHub (pinned to 0132848349)

Solutions

Provide qv (batch, seqlen, nheads, head_dim_qk+head_dim_v packed as the wrapper expects) when using only_qv=True.
Or provide q directly if you have a real query tensor.

Example fix

# before
flash_attn_with_kvcache(only_qv=True, k_cache=kc, v_cache=vc)
# after
flash_attn_with_kvcache(only_qv=True, qv=qv, k_cache=kc, v_cache=vc)

Defensive patterns

Strategy: validation

Validate before calling

if only_qv:
    assert q is not None or qv is not None, "only_qv requires q or qv"

Type guard

def qv_inputs_valid(q, qv, only_qv: bool) -> bool:
    return (not only_qv and q is not None) or (only_qv and (q is not None or qv is not None))

Try / catch

try:
    out = flash_attn_with_kvcache(...)
except ValueError as e:
    if "qv is provided" in str(e):
        raise ValueError("only_qv=True needs qv; pass the packed qv tensor") from None

Prevention

Default qv from q (or vice versa) in your wrapper so one is always derived.
Validate the QV-path inputs in one place before calling FA3.

When it happens

Trigger: flash_attn_with_kvcache(q=None, only_qv=True, qv=None, ...); callers enabling only_qv but passing the query under a different kwarg or forgetting it entirely.

Common situations: Wrappers where qv is optional and defaulted to None; parameter renames during API migration.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/66ca1e9857e63452. Report an issue: GitHub.