{"record":{"id":"165651f15121df92","repo":"sgl-project/sglang","slug":"f-recurrent-kda-state-pool-breaks-the-compiled-str","errorCode":null,"errorMessage":"f\"recurrent_kda state pool breaks the compiled stride contract: slot stride {ssm_states.stride(0)} elements must be a multiple of 16 and the base byte offset {base_bytes} a multiple of 32 (sym_int64(divisibility=16) / assumed_align=32)\"","messagePattern":"f\"recurrent_kda state pool breaks the compiled stride contract: slot stride (.+?) elements must be a multiple of 16 and the base byte offset (.+?) a multiple of 32 \\(sym_int64\\(divisibility=16\\) / assumed_align=32\\)\"","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/linear/kernels/kda_flashinfer.py","lineNumber":118,"sourceCode":"        \"\"\"\n        key = id(ssm_states)\n        if key in self._state_contract_ok:\n            return\n        if ssm_states.dim() != 4:\n            raise ValueError(\n                f\"recurrent_kda needs a [N, HV, V, K] state pool; got \"\n                f\"shape {tuple(ssm_states.shape)}\"\n            )\n        _, hv, v, k = ssm_states.shape\n        if ssm_states.stride()[1:] != (v * k, k, 1):\n            raise ValueError(\n                \"recurrent_kda state inner strides must be compact \"\n                f\"(V*K, K, 1)=({v * k}, {k}, 1); got {ssm_states.stride()[1:]} \"\n                \"(only the slot stride may be non-compact)\"\n            )\n        base_bytes = ssm_states.storage_offset() * ssm_states.element_size()\n        if ssm_states.stride(0) % 16 != 0 or base_bytes % 32 != 0:\n            raise ValueError(\n                \"recurrent_kda state pool breaks the compiled stride contract: \"\n                f\"slot stride {ssm_states.stride(0)} elements must be a multiple \"\n                f\"of 16 and the base byte offset {base_bytes} a multiple of 32 \"\n                \"(sym_int64(divisibility=16) / assumed_align=32)\"\n            )\n        self._state_contract_ok.add(key)\n\n    # ---- gate / beta normalization (shared by decode + verify) ----\n\n    def _prep_gate_params(self, A_log: torch.Tensor, dt_bias: torch.Tensor):\n        # A_log: [1, 1, H, 1] -> [H] fp32; dt_bias: [H*K] (1D) -> fp32. Cached per\n        # layer (constant weights) so this is a dict lookup on the hot path.\n        key = (id(A_log), id(dt_bias))\n        cached = self._gate_cache.get(key)\n        if cached is not None:\n            return cached\n        A_log_fi = A_log.reshape(-1).float().contiguous()\n        dt_bias_fi = (","sourceCodeStart":100,"sourceCodeEnd":136,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/linear/kernels/kda_flashinfer.py#L100-L136","documentation":"The FlashInfer recurrent_kda kernel is compiled with sym_int64(divisibility=16) and assumed_align=32 assumptions: the slot stride (elements) must be divisible by 16 and the pool's base byte offset by 32. Violating this trips an alignment assert in the compiled kernel, so the Python side pre-checks and raises.","triggerScenarios":"Allocating the state pool with a slot stride not a multiple of 16 elements (e.g. odd V*K product) or passing a sliced view whose storage_offset yields a non-32-byte-aligned base.","commonSituations":"Custom cache allocators that pad slots to arbitrary sizes; slicing a shared pool at an unaligned offset; models with head dims whose product V*K is not divisible by 16.","solutions":["Pad the slot stride (dim 0 stride) up to a multiple of 16 elements","Ensure the pool tensor starts at a 32-byte-aligned storage offset (allocate fresh, avoid unaligned slices)","Use the standard SGLang MambaCache allocation which already satisfies the contract"],"exampleFix":"# before\nslot_stride = hv * v * k  # e.g. 8*64*63 not divisible by 16\n# after\nslot_stride = ((hv * v * k + 15) // 16) * 16\npool = torch.empty(num_slots * slot_stride, dtype=torch.float32).view(num_slots, slot_stride)[:, :hv*v*k].view(num_slots, hv, v, k).contiguous()","handlingStrategy":"validation","validationCode":"elem = ssm_states.element_size()\nbase_bytes = ssm_states.storage_offset() * elem\nassert ssm_states.stride(0) % 16 == 0, 'slot stride must be 16-element aligned'\nassert base_bytes % 32 == 0, 'base offset must be 32-byte aligned'","typeGuard":"def meets_kda_alignment(t: torch.Tensor) -> bool:\n    return t.stride(0) % 16 == 0 and (t.storage_offset() * t.element_size()) % 32 == 0","tryCatchPattern":null,"preventionTips":["Pad slot stride to a multiple of 16 elements when allocating custom pools","Allocate pools fresh rather than slicing at unaligned offsets"],"tags":["sglang","kda","alignment","memory-layout","flashinfer"],"backgroundTag":"kernel-alignment-contract-violation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}