sgl-project/sglang · error · ValueError

v_cache must be provided

Error message

v_cache must be provided

What it means

flash_attn_with_kvcache requires the paged KV cache value tensor: if the v_cache argument is None it raises ValueError immediately, before any other validation. v_cache is fundamental to paged attention — without it there is nothing to attend over.

Source

Thrown at python/sglang/kernels/aot/python/sgl_kernel/flash_attn.py:164

            rotary embedding will combine dimensions 0 & rotary_dim / 2, 1 & rotary_dim / 2 + 1
            (i.e. GPT-NeoX style).
        num_splits: int. If > 1, split the key/value into this many chunks along the sequence.
           If num_splits == 1, we don't split the key/value. If num_splits == 0, we use a heuristic
           to automatically determine the number of splits.
           Don't change this unless you know what you are doing.
        return_softmax_lse: bool. Whether to return the logsumexp of the attention scores.
        score_mod [optional]: A callable that takes the attention scores and applies a modification.
        aux_tensors [optional]: Some score_mods will want to read from global aux_tensors. This is how we thread them through to the inner kernel.

    Return:
        out: (batch_size, seqlen, nheads, headdim).
        softmax_lse [optional, if return_softmax_lse=True]: (batch_size, nheads, seqlen). The
            logsumexp of each row of the matrix QK^T * scaling (e.g., log of the softmax
            normalization factor).
    """

    if v_cache is None:
        raise ValueError("v_cache must be provided")
    assert v_cache.stride(-1) == 1, "v_cache must have contiguous last dimension"

    if k_cache is None:
        if not only_qv:
            raise ValueError("k_cache can only be None when only_qv=True")
        if q is not None:
            k_head_size = q.shape[-1]
            k_dtype = q.dtype
            k_device = q.device
        elif k is not None:
            k_head_size = k.shape[-1]
            k_dtype = k.dtype
            k_device = k.device
        else:
            # Fallback: only_qv kernel ignores K values, so a tiny placeholder works.
            k_head_size = 64
            k_dtype = v_cache.dtype
            k_device = v_cache.device

View on GitHub (pinned to 0132848349)

Solutions

  1. Pass a v_cache tensor of shape (num_blocks, num_v_heads_k, block_size, head_dim_v).
  2. Check argument order — many callers pass caches positionally and misorder k_cache/v_cache.
  3. If you meant query-only attention without caches, use flash_attn_func instead of flash_attn_with_kvcache.

Example fix

# before
out = flash_attn_with_kvcache(q=q, k=k, k_cache=k_cache, ...)
# after
out = flash_attn_with_kvcache(q=q, k_cache=k_cache, v_cache=v_cache, ...)
Defensive patterns

Strategy: validation

Validate before calling

assert v_cache is not None and v_cache.stride(-1) == 1, "provide contiguous v_cache"

Type guard

def has_v_cache(v_cache) -> bool:
    return v_cache is not None and v_cache.stride(-1) == 1

Try / catch

try:
    out = flash_attn_with_kvcache(...)
except ValueError as e:
    if "v_cache must be provided" in str(e):
        raise TypeError("paged attention requires a V cache; check pool allocation")

Prevention

When it happens

Trigger: Calling flash_attn_with_kvcache(...) without v_cache (relying on k/k_cache only); passing positional args in the wrong order so v_cache ends up None; wrappers that build cache tensors conditionally and skip V.

Common situations: Adapting MHA-only code to the FA3 paged API; a wrapper forgetting to allocate the V pool (e.g. only_qv path where caller still must supply v_cache).

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9fcfc1eecddb3284. Report an issue: GitHub.