{"record":{"id":"66ca1e9857e63452","repo":"sgl-project/sglang","slug":"q-must-be-provided-unless-qv-is-provided-with-only","errorCode":null,"errorMessage":"q must be provided unless qv is provided with only_qv=True","messagePattern":"q must be provided unless qv is provided with only_qv=True","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/aot/python/sgl_kernel/flash_attn.py","lineNumber":192,"sourceCode":"        elif k is not None:\n            k_head_size = k.shape[-1]\n            k_dtype = k.dtype\n            k_device = k.device\n        else:\n            # Fallback: only_qv kernel ignores K values, so a tiny placeholder works.\n            k_head_size = 64\n            k_dtype = v_cache.dtype\n            k_device = v_cache.device\n        k_shape = (*v_cache.shape[:-1], k_head_size)\n        # The kernel path for only_qv ignores K values, but backend API still requires k tensor.\n        k_cache = torch.empty(k_shape, dtype=k_dtype, device=k_device)\n    assert k_cache.stride(-1) == 1, \"k_cache must have contiguous last dimension\"\n\n    if q is None:\n        if not only_qv:\n            raise ValueError(\"q can only be None when only_qv=True\")\n        if qv is None:\n            raise ValueError(\n                \"q must be provided unless qv is provided with only_qv=True\"\n            )\n        q_shape = (*qv.shape[:-1], k_cache.shape[-1])\n        # The kernel path for only_qv ignores q values, but backend API still requires q tensor.\n        q = torch.empty(q_shape, dtype=qv.dtype, device=qv.device)\n\n    if softmax_scale is None:\n        if only_qv:\n            if qv is None:\n                raise ValueError(\"only_qv=True requires qv to be provided\")\n            softmax_scale = (qv.shape[-1]) ** (-0.5)\n        else:\n            softmax_scale = (q.shape[-1] + (qv.shape[-1] if qv is not None else 0)) ** (\n                -0.5\n            )\n    if cache_seqlens is not None and isinstance(cache_seqlens, int):\n        cache_seqlens = torch.full(\n            (q.shape[0],), cache_seqlens, dtype=torch.int32, device=v_cache.device","sourceCodeStart":174,"sourceCodeEnd":210,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/aot/python/sgl_kernel/flash_attn.py#L174-L210","documentation":"Follow-on guard inside the only_qv path: q is None, only_qv=True, but qv is also None — so the wrapper has no way to infer q's shape/dtype/device to synthesize the dummy q tensor the kernel API requires. Exactly one of q or qv must be provided on the QV path.","triggerScenarios":"flash_attn_with_kvcache(q=None, only_qv=True, qv=None, ...); callers enabling only_qv but passing the query under a different kwarg or forgetting it entirely.","commonSituations":"Wrappers where qv is optional and defaulted to None; parameter renames during API migration.","solutions":["Provide qv (batch, seqlen, nheads, head_dim_qk+head_dim_v packed as the wrapper expects) when using only_qv=True.","Or provide q directly if you have a real query tensor."],"exampleFix":"# before\nflash_attn_with_kvcache(only_qv=True, k_cache=kc, v_cache=vc)\n# after\nflash_attn_with_kvcache(only_qv=True, qv=qv, k_cache=kc, v_cache=vc)","handlingStrategy":"validation","validationCode":"if only_qv:\n    assert q is not None or qv is not None, \"only_qv requires q or qv\"","typeGuard":"def qv_inputs_valid(q, qv, only_qv: bool) -> bool:\n    return (not only_qv and q is not None) or (only_qv and (q is not None or qv is not None))","tryCatchPattern":"try:\n    out = flash_attn_with_kvcache(...)\nexcept ValueError as e:\n    if \"qv is provided\" in str(e):\n        raise ValueError(\"only_qv=True needs qv; pass the packed qv tensor\") from None","preventionTips":["Default qv from q (or vice versa) in your wrapper so one is always derived.","Validate the QV-path inputs in one place before calling FA3."],"tags":["sglang","flash-attention","missing-argument","validation"],"backgroundTag":"missing-required-argument","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}