sgl-project/sglang · error · ValueError

Validate failed: unsupported tensor shape: {t.shape}.

Error message

Validate failed: unsupported tensor shape: {t.shape}.

What it means

validate_x requires activations shaped exactly (B, S, D): a 3D batch/sequence/hidden tensor. The CuTe DSL kernel tiles the D dimension and iterates B*S rows, so other ranks or mismatched dims are rejected up front.

Source

Thrown at python/sglang/kernels/ops/diffusion/norm/scale_residual_norm_cutedsl.py:196

        tNrN = norm(tNrN, tWrW, tBrB)
        # Compute: value = value * (1 + <scale>) + <shift>
        value = tNrN.load()
        copy_if(tSCgSC, tSCrSC)  # gmem -> rmem
        copy_if(tSHgSH, tSHrSH)  # gmem -> rmem
        if cutlass.const_expr(isinstance(tSCrSC, cute.Tensor)):
            value = value * (1 + tSCrSC.load())
        if cutlass.const_expr(isinstance(tSHrSH, cute.Tensor)):
            value = value + tSHrSH.load()
        # Store: y
        tYrY.store(value.to(tYrY.element_type))
        copy_if(tYrY, tYgY)  # rmem -> gmem


def validate_x(t: torch.Tensor, B: int, S: int, D: int):
    if t.dtype not in (torch.float16, torch.bfloat16, torch.float32):
        raise ValueError(f"Validate failed: unsupported dtype: {t.dtype}")
    if t.shape != (B, S, D):
        raise ValueError(f"Validate failed: unsupported tensor shape: {t.shape}.")
    if t.stride()[-1] != 1:
        raise ValueError("Validate failed: not contiguous on dim D.")


def validate_weight_bias(t: Optional[torch.Tensor], D: int):
    if t is None:
        return
    if t.dtype not in (torch.float16, torch.bfloat16, torch.float32):
        raise ValueError(f"Validate failed: unsupported dtype: {t.dtype}")
    if t.shape != (D,):
        raise ValueError(f"Validate failed: unsupported tensor shape: {t.shape}.")
    if t.stride()[-1] != 1:
        raise ValueError("Validate failed: not contiguous on dim D.")


def validate_scale_shift(t: torch.Tensor, B: int, S: int, D: int):
    if t.dtype not in (torch.float16, torch.bfloat16, torch.float32):
        raise ValueError(f"Validate failed: unsupported dtype: {t.dtype}")

View on GitHub (pinned to 0132848349)

Solutions

1. Reshape/unsqueeze the activation to exactly (B, S, D) before the call
2. Make sure the B, S, D you pass to validators/ops come from x.shape itself, not separate bookkeeping

Example fix

# before
y = fused_norm_scale_shift(x_2d, ...)  # x_2d: (S, D)
# after
y = fused_norm_scale_shift(x_2d.unsqueeze(0), ...)  # (1, S, D)

assert x.ndim == 3 and tuple(x.shape) == (B, S, D), f"expected (B,S,D), got {tuple(x.shape)}"

def is_bsd(t: torch.Tensor, B: int, S: int, D: int) -> bool:
    return t.ndim == 3 and tuple(t.shape) == (B, S, D)

When it happens

Trigger: Passing a 2D (S, D) tensor, a 4D tensor, or a 3D tensor whose dims don't match the B/S/D the caller declared to fused_norm_scale_shift / fused_scale_residual_norm_scale_shift.

Common situations: Feeding unbatched 2D activations, forgetting to unsqueeze a batch dim, or a mismatch between declared BSD and actual tensor shape after slicing/padding.

Related errors

• Validate failed: unsupported dtype: {t.dtype}
• O tensor must match dtype
• timestep must have shape [B, S, 9 * D]
• Validate failed: not contiguous on dim D.
• Validate failed: S({S}) must be divisible by F({F}).

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7a12596b5122fbb2. Report an issue: GitHub.