{"record":{"id":"27160e26316b8f51","repo":"sgl-project/sglang","slug":"kda-prefill-is-the-inference-forward-path-cp-cont","errorCode":null,"errorMessage":"kda_prefill is the inference forward path: cp_context, and disable_recompute are training-side knobs it does not implement","messagePattern":"kda_prefill is the inference forward path: cp_context, and disable_recompute are training-side knobs it does not implement","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/linear/kda_ptx_prefill/__init__.py","lineNumber":114,"sourceCode":"      beta [B,T,H] bf16 (fp32 also accepted; widened to fp32 in the ext).\n      use_qk_l2norm_in_kernel=True accepts raw q/k and applies FLA-compatible\n        L2 normalization (eps=1e-6, bf16 rounding) in the CUDA tile loads.\n      use_beta_sigmoid_in_kernel=True accepts beta logits and fuses sigmoid.\n      cu_seqlens: host values are needed for the kernel's per-sequence piece\n        table -- pass cu_seqlens_cpu to avoid the D2H sync; chunk_indices is\n        accepted and ignored (the kernel derives its own piece table).\n      initial_state [N,H,128,128] fp32 or None (zeros).\n      return_intermediate_states=True returns dense fp32 chunk-boundary states\n        [1, NT, H, 128, 128] at tuple index 10.\n\n    Returns the fla-shaped 12-tuple: (o [B,T,H,128] bf16, final_state\n    [N,H,128,128] fp32 or None, then Nones, ..., h, initial_state).\n    \"\"\"\n    assert (\n        chunk_size == CHUNK\n    ), f\"kda_prefill supports chunk_size={CHUNK} only, got {chunk_size}\"\n    if cp_context is not None or disable_recompute:\n        raise NotImplementedError(\n            \"kda_prefill is the inference forward path: cp_context, \"\n            \"and disable_recompute are training-side knobs it does not implement\"\n        )\n    if allow_neg_eigval and use_beta_sigmoid_in_kernel:\n        raise NotImplementedError(\n            \"allow_neg_eigval=True requires 2*sigmoid(beta), which is not \"\n            \"implemented by the fused beta path; pass pre-activated beta with \"\n            \"use_beta_sigmoid_in_kernel=False\"\n        )\n    if state_v_first and initial_state is not None:\n        # [V,K]-layout state: pure transpose (K==V==128), exact, ~us/call\n        initial_state = initial_state.transpose(-1, -2).contiguous()\n    assert (\n        q.dim() == 4 and q.shape[-1] == K and v.shape[-1] == K\n    ), f\"expected [B,T,H,{K}] q/k/v, got q={tuple(q.shape)} v={tuple(v.shape)}\"\n    B, T, H, _ = q.shape\n\n    cu_cpu = None","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/linear/kda_ptx_prefill/__init__.py#L96-L132","documentation":"kda_prefill (kda_ptx_prefill) is the inference-only forward. The public entry asserts that training-side context-parallelism and recompute knobs (cp_context, disable_recompute) are not passed. Passing either one raises NotImplementedError immediately.","triggerScenarios":"Calling the kda_prefill forward with cp_context not None or disable_recompute=True (values copied from a training code path / fla-style chunk_kda API).","commonSituations":"Porting a training implementation (e.g. FLA chunk_kda with context parallelism) into SGLang serving and reusing the same kwargs; a model definition shared between train and inference passing all knobs unconditionally.","solutions":["Pass cp_context=None and disable_recompute=False (or omit them) when calling the inference prefill","Use the dedicated training-side kernel entry (fla/training path) for cp / recompute semantics"],"exampleFix":"// before\nkda_prefill(q, k, v, ..., cp_context=cp_ctx, disable_recompute=True)\n// after\nkda_prefill(q, k, v, ..., cp_context=None, disable_recompute=False)","handlingStrategy":"validation","validationCode":"assert cp_context is None and not disable_recompute, \\\n    'kda_prefill is inference-only: drop training knobs'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep separate call sites for training and inference KDA paths","Omit training kwargs entirely on the inference path"],"tags":["kda","training-vs-inference","not-implemented","invalid-argument"],"backgroundTag":"unsupported-argument-for-code-path","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}