sgl-project/sglang · error · ValueError

`A_log`/`dt_bias` must be 1D tensors.

Error message

`A_log`/`dt_bias` must be 1D tensors.

What it means

A_log and dt_bias are per-value-head 1D parameter vectors (shape (HV,)) used to compute the recurrent decay a = exp(A_log) and the dt bias in the fused decode kernel. The wrapper raises when either tensor is not rank-1, e.g. when the full 2D weight matrix or a batched copy is passed.

Source

Thrown at python/sglang/kernels/ops/attention/fla/fused_recurrent.py:293

    initial_state: torch.Tensor,
    out: torch.Tensor,
    ssm_state_indices: torch.Tensor,
    use_qk_l2norm_in_kernel: bool = False,
) -> tuple[torch.Tensor, torch.Tensor]:
    if mixed_qkv.ndim != 2:
        raise ValueError(
            f"`mixed_qkv` must be a 2D tensor (got ndim={mixed_qkv.ndim})."
        )
    if mixed_qkv.stride(-1) != 1:
        raise ValueError("`mixed_qkv` must be contiguous in the last dim.")
    if a.ndim != 2 or b.ndim != 2:
        raise ValueError(
            f"`a` and `b` must be 2D tensors (got a.ndim={a.ndim}, b.ndim={b.ndim})."
        )
    if a.stride(-1) != 1 or b.stride(-1) != 1:
        raise ValueError("`a`/`b` must be contiguous in the last dim.")
    if A_log.ndim != 1 or dt_bias.ndim != 1:
        raise ValueError("`A_log`/`dt_bias` must be 1D tensors.")
    if A_log.stride(0) != 1 or dt_bias.stride(0) != 1:
        raise ValueError("`A_log`/`dt_bias` must be contiguous.")
    if ssm_state_indices.ndim != 1:
        raise ValueError(
            f"`ssm_state_indices` must be 1D for packed decode (got ndim={ssm_state_indices.ndim})."
        )
    if not out.is_contiguous():
        raise ValueError("`out` must be contiguous.")

    dev = mixed_qkv.device
    if any(
        t.device != dev
        for t in (a, b, A_log, dt_bias, initial_state, out, ssm_state_indices)
    ):
        raise ValueError("All inputs must be on the same device.")

    B = mixed_qkv.shape[0]
    if a.shape[0] != B or b.shape[0] != B:

View on GitHub (pinned to 0132848349)

Solutions

  1. Squeeze to 1D before the call: A_log = A_log.squeeze(), dt_bias = dt_bias.squeeze() (or .reshape(-1))
  2. Fix the weight loader to store these params as flat (HV,) tensors matching the model definition

Example fix

# before
out, s = fused_recurrent_gated_delta_rule_packed_decode(..., A_log=A_log, dt_bias=dt_bias)  # (1, HV)
# after
out, s = fused_recurrent_gated_delta_rule_packed_decode(..., A_log=A_log.reshape(-1), dt_bias=dt_bias.reshape(-1))
Defensive patterns

Strategy: validation

Validate before calling

A_log = A_log.reshape(-1) if A_log.ndim != 1 else A_log
dt_bias = dt_bias.reshape(-1) if dt_bias.ndim != 1 else dt_bias

Type guard

def flat_1d(t: torch.Tensor) -> bool:
    return t.ndim == 1

Prevention

When it happens

Trigger: Passing A_log with shape (1, HV), (B, HV), or (HV, 1) — typical when parameters are stored with an extra dim (Einsum-style params, bmm-ready layouts) or batched per token.

Common situations: Model checkpoints that store A_log as (num_heads, 1) or (1, num_heads); vLLM/sglang weight loaders that insert a leading dim; test code broadcasting params to batch shape.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3ce5b7f00a595eda. Report an issue: GitHub.