{"record":{"id":"9fcfc1eecddb3284","repo":"sgl-project/sglang","slug":"v-cache-must-be-provided","errorCode":null,"errorMessage":"v_cache must be provided","messagePattern":"v_cache must be provided","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/aot/python/sgl_kernel/flash_attn.py","lineNumber":164,"sourceCode":"            rotary embedding will combine dimensions 0 & rotary_dim / 2, 1 & rotary_dim / 2 + 1\n            (i.e. GPT-NeoX style).\n        num_splits: int. If > 1, split the key/value into this many chunks along the sequence.\n           If num_splits == 1, we don't split the key/value. If num_splits == 0, we use a heuristic\n           to automatically determine the number of splits.\n           Don't change this unless you know what you are doing.\n        return_softmax_lse: bool. Whether to return the logsumexp of the attention scores.\n        score_mod [optional]: A callable that takes the attention scores and applies a modification.\n        aux_tensors [optional]: Some score_mods will want to read from global aux_tensors. This is how we thread them through to the inner kernel.\n\n    Return:\n        out: (batch_size, seqlen, nheads, headdim).\n        softmax_lse [optional, if return_softmax_lse=True]: (batch_size, nheads, seqlen). The\n            logsumexp of each row of the matrix QK^T * scaling (e.g., log of the softmax\n            normalization factor).\n    \"\"\"\n\n    if v_cache is None:\n        raise ValueError(\"v_cache must be provided\")\n    assert v_cache.stride(-1) == 1, \"v_cache must have contiguous last dimension\"\n\n    if k_cache is None:\n        if not only_qv:\n            raise ValueError(\"k_cache can only be None when only_qv=True\")\n        if q is not None:\n            k_head_size = q.shape[-1]\n            k_dtype = q.dtype\n            k_device = q.device\n        elif k is not None:\n            k_head_size = k.shape[-1]\n            k_dtype = k.dtype\n            k_device = k.device\n        else:\n            # Fallback: only_qv kernel ignores K values, so a tiny placeholder works.\n            k_head_size = 64\n            k_dtype = v_cache.dtype\n            k_device = v_cache.device","sourceCodeStart":146,"sourceCodeEnd":182,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/aot/python/sgl_kernel/flash_attn.py#L146-L182","documentation":"flash_attn_with_kvcache requires the paged KV cache value tensor: if the v_cache argument is None it raises ValueError immediately, before any other validation. v_cache is fundamental to paged attention — without it there is nothing to attend over.","triggerScenarios":"Calling flash_attn_with_kvcache(...) without v_cache (relying on k/k_cache only); passing positional args in the wrong order so v_cache ends up None; wrappers that build cache tensors conditionally and skip V.","commonSituations":"Adapting MHA-only code to the FA3 paged API; a wrapper forgetting to allocate the V pool (e.g. only_qv path where caller still must supply v_cache).","solutions":["Pass a v_cache tensor of shape (num_blocks, num_v_heads_k, block_size, head_dim_v).","Check argument order — many callers pass caches positionally and misorder k_cache/v_cache.","If you meant query-only attention without caches, use flash_attn_func instead of flash_attn_with_kvcache."],"exampleFix":"# before\nout = flash_attn_with_kvcache(q=q, k=k, k_cache=k_cache, ...)\n# after\nout = flash_attn_with_kvcache(q=q, k_cache=k_cache, v_cache=v_cache, ...)","handlingStrategy":"validation","validationCode":"assert v_cache is not None and v_cache.stride(-1) == 1, \"provide contiguous v_cache\"","typeGuard":"def has_v_cache(v_cache) -> bool:\n    return v_cache is not None and v_cache.stride(-1) == 1","tryCatchPattern":"try:\n    out = flash_attn_with_kvcache(...)\nexcept ValueError as e:\n    if \"v_cache must be provided\" in str(e):\n        raise TypeError(\"paged attention requires a V cache; check pool allocation\")","preventionTips":["Allocate both K and V pools together in KV-cache managers.","Keyword-arg all cache tensors to avoid positional misordering."],"tags":["sglang","flash-attention","kv-cache","missing-argument","validation"],"backgroundTag":"missing-required-argument","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}