{"record":{"id":"f0e8c28e3c1dd042","repo":"sgl-project/sglang","slug":"q-can-only-be-none-when-only-qv-true","errorCode":null,"errorMessage":"q can only be None when only_qv=True","messagePattern":"q can only be None when only_qv=True","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/aot/python/sgl_kernel/flash_attn.py","lineNumber":190,"sourceCode":"            k_dtype = q.dtype\n            k_device = q.device\n        elif k is not None:\n            k_head_size = k.shape[-1]\n            k_dtype = k.dtype\n            k_device = k.device\n        else:\n            # Fallback: only_qv kernel ignores K values, so a tiny placeholder works.\n            k_head_size = 64\n            k_dtype = v_cache.dtype\n            k_device = v_cache.device\n        k_shape = (*v_cache.shape[:-1], k_head_size)\n        # The kernel path for only_qv ignores K values, but backend API still requires k tensor.\n        k_cache = torch.empty(k_shape, dtype=k_dtype, device=k_device)\n    assert k_cache.stride(-1) == 1, \"k_cache must have contiguous last dimension\"\n\n    if q is None:\n        if not only_qv:\n            raise ValueError(\"q can only be None when only_qv=True\")\n        if qv is None:\n            raise ValueError(\n                \"q must be provided unless qv is provided with only_qv=True\"\n            )\n        q_shape = (*qv.shape[:-1], k_cache.shape[-1])\n        # The kernel path for only_qv ignores q values, but backend API still requires q tensor.\n        q = torch.empty(q_shape, dtype=qv.dtype, device=qv.device)\n\n    if softmax_scale is None:\n        if only_qv:\n            if qv is None:\n                raise ValueError(\"only_qv=True requires qv to be provided\")\n            softmax_scale = (qv.shape[-1]) ** (-0.5)\n        else:\n            softmax_scale = (q.shape[-1] + (qv.shape[-1] if qv is not None else 0)) ** (\n                -0.5\n            )\n    if cache_seqlens is not None and isinstance(cache_seqlens, int):","sourceCodeStart":172,"sourceCodeEnd":208,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/aot/python/sgl_kernel/flash_attn.py#L172-L208","documentation":"Symmetric guard on the query side: q may only be None when only_qv=True, where a dummy q is materialized from qv's shape and k_cache's head dim because the backend API still requires a q tensor even though the kernel ignores its values. A None q on the standard path is a caller bug.","triggerScenarios":"flash_attn_with_kvcache(q=None, ...) without only_qv=True; only_qv=True but qv also None (then the next guard fires); misordered keyword args so q receives None.","commonSituations":"QV-only attention models (e.g. MLA-like paths) integrated with the FA3 wrapper; refactors that stopped materializing q without setting the flag.","solutions":["Set only_qv=True and supply qv (and v_cache) for the query-value path.","Or pass a real q tensor with shape (..., nheads, head_dim)."],"exampleFix":"# before\nflash_attn_with_kvcache(q=None, k_cache=kc, v_cache=vc)\n# after\nflash_attn_with_kvcache(q=None, only_qv=True, qv=qv, k_cache=kc, v_cache=vc)","handlingStrategy":"validation","validationCode":"if q is None:\n    assert only_qv is True and qv is not None","typeGuard":"def q_valid(q, only_qv: bool, qv) -> bool:\n    return q is not None or (only_qv and qv is not None)","tryCatchPattern":"try:\n    out = flash_attn_with_kvcache(...)\nexcept ValueError as e:\n    if \"q can only be None\" in str(e):\n        out = flash_attn_with_kvcache(..., only_qv=True, qv=qv)","preventionTips":["Keep (only_qv, q, qv) as a single config tuple in attention wrappers so flags and tensors stay consistent.","Add unit tests covering both q-provided and qv-only paths."],"tags":["sglang","flash-attention","invalid-argument","validation"],"backgroundTag":"invalid-argument-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}