{"record":{"id":"abc9cf5de295bcef","repo":"sgl-project/sglang","slug":"f-recurrent-kda-state-inner-strides-must-be-compac","errorCode":null,"errorMessage":"f\"recurrent_kda state inner strides must be compact (V*K, K, 1)=({v * k}, {k}, 1); got {ssm_states.stride()[1:]} (only the slot stride may be non-compact)\"","messagePattern":"f\"recurrent_kda state inner strides must be compact \\(V\\*K, K, 1\\)=\\((.+?), (.+?), 1\\); got (.+?) \\(only the slot stride may be non-compact\\)\"","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/linear/kernels/kda_flashinfer.py","lineNumber":111,"sourceCode":"        and ``assumed_align=32`` (flashinfer ``kda_kernels/recurrent_kda.py``):\n        the slot stride is free — which is what lets the envelope-strided pools\n        (unified memory / page-major layout, slot stride = per-slot envelope\n        pitch) be passed in and updated IN PLACE on the cu_seqlens path — but\n        the inner strides are compiled-in constants and the divisibility /\n        alignment are hard assumptions. A pool violating them would mis-address\n        state in-kernel without any error; fail loudly here instead.\n        \"\"\"\n        key = id(ssm_states)\n        if key in self._state_contract_ok:\n            return\n        if ssm_states.dim() != 4:\n            raise ValueError(\n                f\"recurrent_kda needs a [N, HV, V, K] state pool; got \"\n                f\"shape {tuple(ssm_states.shape)}\"\n            )\n        _, hv, v, k = ssm_states.shape\n        if ssm_states.stride()[1:] != (v * k, k, 1):\n            raise ValueError(\n                \"recurrent_kda state inner strides must be compact \"\n                f\"(V*K, K, 1)=({v * k}, {k}, 1); got {ssm_states.stride()[1:]} \"\n                \"(only the slot stride may be non-compact)\"\n            )\n        base_bytes = ssm_states.storage_offset() * ssm_states.element_size()\n        if ssm_states.stride(0) % 16 != 0 or base_bytes % 32 != 0:\n            raise ValueError(\n                \"recurrent_kda state pool breaks the compiled stride contract: \"\n                f\"slot stride {ssm_states.stride(0)} elements must be a multiple \"\n                f\"of 16 and the base byte offset {base_bytes} a multiple of 32 \"\n                \"(sym_int64(divisibility=16) / assumed_align=32)\"\n            )\n        self._state_contract_ok.add(key)\n\n    # ---- gate / beta normalization (shared by decode + verify) ----\n\n    def _prep_gate_params(self, A_log: torch.Tensor, dt_bias: torch.Tensor):\n        # A_log: [1, 1, H, 1] -> [H] fp32; dt_bias: [H*K] (1D) -> fp32. Cached per","sourceCodeStart":93,"sourceCodeEnd":129,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/linear/kernels/kda_flashinfer.py#L93-L129","documentation":"Beyond shape, recurrent_kda requires the inner (HV, V, K) dims of the state pool to be compactly strided (V*K, K, 1). Only the slot (first) dimension stride may be padded. Non-contiguous inner strides would silently corrupt in-kernel state, so it fails loudly.","triggerScenarios":"Passing a state pool that is a transpose/slice with non-unit inner stride, e.g. state.transpose(-1,-2) of the SGLang layout, or a view over a KV-interleaved buffer.","commonSituations":"Custom kernels or tests that materialize states in the NVIDIA [B,H,K,V] layout and pass it directly to FlashInfer decode without a .contiguous() round-trip.","solutions":["Materialize the pool as [N, HV, V, K] and call .contiguous() so inner strides are (V*K, K, 1)","If converting from a vendor K-major layout, transpose then contiguous() before decode","Keep padding only on the slot (dim 0) stride"],"exampleFix":"# before\nssm_states = vendor_state  # [N, HV, K, V] strides\n# after\nssm_states = vendor_state.transpose(-1, -2).contiguous()  # [N, HV, V, K] compact","handlingStrategy":"validation","validationCode":"_, hv, v, k = ssm_states.shape\nassert tuple(ssm_states.stride()[1:]) == (v * k, k, 1), f'non-compact inner strides: {ssm_states.stride()}'","typeGuard":"def has_compact_inner_strides(t: torch.Tensor) -> bool:\n    if t.dim() != 4: return False\n    _, _, v, k = t.shape\n    return tuple(t.stride()[1:]) == (v * k, k, 1)","tryCatchPattern":null,"preventionTips":["Call .contiguous() on any transposed state before decode","Never pass vendor K-major layouts directly; convert then contiguous"],"tags":["sglang","kda","tensor-stride","contiguity","validation"],"backgroundTag":"tensor-stride-contract-violation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}