{"record":{"id":"3ce5b7f00a595eda","repo":"sgl-project/sglang","slug":"a-log-dt-bias-must-be-1d-tensors","errorCode":null,"errorMessage":"`A_log`/`dt_bias` must be 1D tensors.","messagePattern":"`A_log`/`dt_bias` must be 1D tensors\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fla/fused_recurrent.py","lineNumber":293,"sourceCode":"    initial_state: torch.Tensor,\n    out: torch.Tensor,\n    ssm_state_indices: torch.Tensor,\n    use_qk_l2norm_in_kernel: bool = False,\n) -> tuple[torch.Tensor, torch.Tensor]:\n    if mixed_qkv.ndim != 2:\n        raise ValueError(\n            f\"`mixed_qkv` must be a 2D tensor (got ndim={mixed_qkv.ndim}).\"\n        )\n    if mixed_qkv.stride(-1) != 1:\n        raise ValueError(\"`mixed_qkv` must be contiguous in the last dim.\")\n    if a.ndim != 2 or b.ndim != 2:\n        raise ValueError(\n            f\"`a` and `b` must be 2D tensors (got a.ndim={a.ndim}, b.ndim={b.ndim}).\"\n        )\n    if a.stride(-1) != 1 or b.stride(-1) != 1:\n        raise ValueError(\"`a`/`b` must be contiguous in the last dim.\")\n    if A_log.ndim != 1 or dt_bias.ndim != 1:\n        raise ValueError(\"`A_log`/`dt_bias` must be 1D tensors.\")\n    if A_log.stride(0) != 1 or dt_bias.stride(0) != 1:\n        raise ValueError(\"`A_log`/`dt_bias` must be contiguous.\")\n    if ssm_state_indices.ndim != 1:\n        raise ValueError(\n            f\"`ssm_state_indices` must be 1D for packed decode (got ndim={ssm_state_indices.ndim}).\"\n        )\n    if not out.is_contiguous():\n        raise ValueError(\"`out` must be contiguous.\")\n\n    dev = mixed_qkv.device\n    if any(\n        t.device != dev\n        for t in (a, b, A_log, dt_bias, initial_state, out, ssm_state_indices)\n    ):\n        raise ValueError(\"All inputs must be on the same device.\")\n\n    B = mixed_qkv.shape[0]\n    if a.shape[0] != B or b.shape[0] != B:","sourceCodeStart":275,"sourceCodeEnd":311,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fla/fused_recurrent.py#L275-L311","documentation":"A_log and dt_bias are per-value-head 1D parameter vectors (shape (HV,)) used to compute the recurrent decay a = exp(A_log) and the dt bias in the fused decode kernel. The wrapper raises when either tensor is not rank-1, e.g. when the full 2D weight matrix or a batched copy is passed.","triggerScenarios":"Passing A_log with shape (1, HV), (B, HV), or (HV, 1) — typical when parameters are stored with an extra dim (Einsum-style params, bmm-ready layouts) or batched per token.","commonSituations":"Model checkpoints that store A_log as (num_heads, 1) or (1, num_heads); vLLM/sglang weight loaders that insert a leading dim; test code broadcasting params to batch shape.","solutions":["Squeeze to 1D before the call: A_log = A_log.squeeze(), dt_bias = dt_bias.squeeze() (or .reshape(-1))","Fix the weight loader to store these params as flat (HV,) tensors matching the model definition"],"exampleFix":"# before\nout, s = fused_recurrent_gated_delta_rule_packed_decode(..., A_log=A_log, dt_bias=dt_bias)  # (1, HV)\n# after\nout, s = fused_recurrent_gated_delta_rule_packed_decode(..., A_log=A_log.reshape(-1), dt_bias=dt_bias.reshape(-1))","handlingStrategy":"validation","validationCode":"A_log = A_log.reshape(-1) if A_log.ndim != 1 else A_log\ndt_bias = dt_bias.reshape(-1) if dt_bias.ndim != 1 else dt_bias","typeGuard":"def flat_1d(t: torch.Tensor) -> bool:\n    return t.ndim == 1","tryCatchPattern":null,"preventionTips":["Store A_log/dt_bias as flat (HV,) parameters in the module definition","Add a weight-loader assertion that these params load as 1D"],"tags":["fla","fused-recurrent","shape-validation","mamba-params"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}