{"record":{"id":"aec7810a64245092","repo":"sgl-project/sglang","slug":"g-and-beta-must-cover-every-q-token","errorCode":null,"errorMessage":"g and beta must cover every q token","messagePattern":"g and beta must cover every q token","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/helion/kda_prefill.py","lineNumber":1327,"sourceCode":"    initial_state: torch.Tensor | None = None,\n    initial_state_indices: torch.Tensor | None = None,\n    use_qk_l2norm_in_kernel: bool = False,\n    cu_seqlens: torch.Tensor | None = None,\n    A_log: torch.Tensor | None = None,\n    dt_bias: torch.Tensor | None = None,\n    lower_bound: float | None = None,\n    output_intermediate_states: bool = False,\n    **kwargs: object,\n) -> torch.Tensor | tuple[torch.Tensor, torch.Tensor]:\n    \"\"\"Match the public forward contract of SGLang's Triton ``chunk_kda``.\"\"\"\n    if scale is None:\n        scale = k.shape[-1] ** -0.5\n    if initial_state is None or initial_state_indices is None:\n        raise ValueError(\"KDA prefill requires an indexed initial-state pool\")\n\n    num_tokens = q.shape[1]\n    if g.shape[1] < num_tokens or beta.shape[1] < num_tokens:\n        raise ValueError(\"g and beta must cover every q token\")\n    g = g[:, :num_tokens]\n    beta = beta[:, :num_tokens]\n    if num_tokens == 1:\n        # Tracing constant-folds size-one dimensions, but the resulting kernel\n        # can share a cache entry with longer inputs. Keep T=1 on Triton so a\n        # short first request cannot specialize later Helion calls incorrectly.\n        return triton_chunk_kda(\n            q=q,\n            k=k,\n            v=v,\n            g=g,\n            beta=beta,\n            scale=scale,\n            initial_state=initial_state,\n            initial_state_indices=initial_state_indices,\n            use_qk_l2norm_in_kernel=use_qk_l2norm_in_kernel,\n            cu_seqlens=cu_seqlens,\n            A_log=A_log,","sourceCodeStart":1309,"sourceCodeEnd":1345,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/helion/kda_prefill.py#L1309-L1345","documentation":"chunk_kda uses q.shape[1] as the token count and requires the decay gates g and beta to have at least that many tokens along dim 1 (they are then trimmed with g[:, :num_tokens]). The error fires when g or beta is shorter than q, i.e. the gate projections do not cover all query tokens.","triggerScenarios":"Calling chunk_kda with g or beta computed over fewer tokens than q — e.g. gates sliced to a previous chunk length, a truncated gate projection, or varlen tensors packed with a different total than q.","commonSituations":"Chunked prefill where the gate projection lags one chunk behind q; scheduler merging requests but recomputing gates only for part of them; off-by-one slicing g[:, :T-1] when preparing inputs.","solutions":["Recompute or re-slice g and beta so g.shape[1] >= q.shape[1] and beta.shape[1] >= q.shape[1]","Check the upstream projection ran on the same token set as q/k/v (same cu_seqlens)","In chunked prefill, make sure gate projections are re-run for the full extended chunk, not cached from a shorter one"],"exampleFix":"// before\ng = g[:, :prev_len]  # stale chunk length\nchunk_kda(q, k, v, g, beta, ...)\n// after\ng = compute_gates(x_full)  # covers all q tokens\nchunk_kda(q, k, v, g, beta, ...)","handlingStrategy":"validation","validationCode":"num_tokens = q.shape[1]\nassert g.shape[1] >= num_tokens and beta.shape[1] >= num_tokens, (\n    g.shape, beta.shape, q.shape)\ng, beta = g[:, :num_tokens], beta[:, :num_tokens]","typeGuard":"def gates_cover_q(q, g, beta) -> bool:\n    return g.shape[1] >= q.shape[1] and beta.shape[1] >= q.shape[1]","tryCatchPattern":null,"preventionTips":["Compute gates from the same token set as q/k/v every step","In chunked prefill, never reuse gate tensors from a shorter chunk"],"tags":["kda","helion","prefill","length-mismatch"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}