sgl-project/sglang · error · ValueError

`write_pos` must be a 1D int32 tensor.

Error message

`write_pos` must be a 1D int32 tensor.

What it means

write_pos — the per-row ring write position for the replaySSM decode kernel — must be a 1D tensor of dtype torch.int32. Wrong rank (scalar/2D) or dtype (int64/int16) is rejected because the Triton kernel loads it directly as int32 offsets.

Source

Thrown at python/sglang/kernels/ops/attention/fla/fused_recurrent_linear_replayssm.py:490

    Allocates nothing persistent: the caller owns the ring tensors and is
    responsible for advancing / resetting ``write_pos`` (e.g. ``(write_pos+1) %
    L`` after each step).  This is a STANDALONE kernel; the memory-pool / cache
    integration is a later phase.
    """
    if mixed_qkv.ndim != 2:
        raise ValueError(f"`mixed_qkv` must be 2D (got ndim={mixed_qkv.ndim}).")
    if mixed_qkv.stride(-1) != 1:
        raise ValueError("`mixed_qkv` must be contiguous in the last dim.")
    if b.ndim != 2:
        raise ValueError(f"`b` must be 2D (got b.ndim={b.ndim}).")
    if A_log.ndim != 1:
        raise ValueError("`A_log` must be a 1D tensor.")
    if initial_state.ndim != 4:
        raise ValueError(f"`initial_state` must be 4D (got ndim={initial_state.ndim}).")
    if not out.is_contiguous():
        raise ValueError("`out` must be contiguous.")
    if write_pos.ndim != 1 or write_pos.dtype != torch.int32:
        raise ValueError("`write_pos` must be a 1D int32 tensor.")
    if force_flush is not None and (
        force_flush.ndim != 1 or force_flush.dtype != torch.int32
    ):
        raise ValueError("`force_flush` must be a 1D int32 tensor or None.")

    B = mixed_qkv.shape[0]
    num_state_slots, HV, V, K = initial_state.shape
    qkv_dim = mixed_qkv.shape[1]
    q_dim = (qkv_dim - HV * V) // 2
    if q_dim <= 0 or q_dim % K != 0:
        raise ValueError(
            f"Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V={V}, K={K}."
        )
    H = q_dim // K
    if H <= 0 or HV % H != 0:
        raise ValueError(
            f"Invalid head config inferred from mixed_qkv: H={H}, HV={HV}."
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Create/convert as: write_pos = write_pos.to(torch.int32) and ensure .ndim == 1 (one per row/slot)
2. Pin dtype explicitly at allocation: torch.zeros(N, dtype=torch.int32, device=dev)
3. Also make force_flush 1D int32 if used

Example fix

// before
write_pos = torch.arange(N, device=dev)  # int64
// after
write_pos = torch.arange(N, device=dev, dtype=torch.int32)

write_pos = write_pos.to(torch.int32).reshape(-1)
assert write_pos.ndim == 1 and write_pos.dtype == torch.int32

def valid_write_pos(t: torch.Tensor) -> bool:
    return t.ndim == 1 and t.dtype == torch.int32

When it happens

Trigger: Passing a Python int, an int64 tensor (e.g. from torch.arange default), or a per-head [N, HV] position tensor.

Common situations: Creating positions with torch.arange without dtype=torch.int32; advancing write_pos with arithmetic that promotes to int64; graph-capture tests reusing stale dtype buffers.

Related errors

• `mixed_qkv` must be 2D (got ndim={mixed_qkv.ndim}).
• `mixed_qkv` must be contiguous in the last dim.
• `b` must be 2D (got b.ndim={b.ndim}).
• `A_log` must be a 1D tensor.
• `initial_state` must be 4D (got ndim={initial_state.ndim}).

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e050cf7ffe859c83. Report an issue: GitHub.