sgl-project/sglang · error · ValueError

timestep last dim must equal 9 * hidden

Error message

timestep last dim must equal 9 * hidden

What it means

The timestep embedding's last dimension must be exactly total_params * hidden = 9 * D, matching the scale_shift_table's [9, D] shape. A mismatch (e.g. 6*D from a standard AdaLN projection, or a different hidden size) raises this ValueError.

Source

Thrown at python/sglang/kernels/ops/diffusion/modulate/ltx2_ada_values_triton.py:162

    if not timestep.is_cuda or timestep.dtype != torch.bfloat16:
        raise ValueError("timestep must be a CUDA bfloat16 tensor")
    if not timestep.is_contiguous():
        raise ValueError("timestep must be contiguous")
    if scale_shift_table.ndim != 2 or scale_shift_table.shape[0] != 9:
        raise ValueError("scale_shift_table must have shape [9, D]")
    if (
        not scale_shift_table.is_cuda
        or scale_shift_table.dtype not in (torch.bfloat16, torch.float32)
        or scale_shift_table.stride(-1) != 1
    ):
        raise ValueError(
            "scale_shift_table must be CUDA, bf16/fp32, last-dim contiguous"
        )

    total_params = int(scale_shift_table.shape[0])
    hidden = int(scale_shift_table.shape[1])
    if hidden <= 0 or timestep.shape[-1] != total_params * hidden:
        raise ValueError("timestep last dim must equal 9 * hidden")
    if hidden % 256 != 0 or hidden > 8192:
        raise ValueError("hidden size is outside the supported LTX2 fast-path range")

    batch, seq, _ = timestep.shape
    rows = int(batch * seq)
    # Each returned output is a disjoint, contiguous view, so one allocation
    # avoids nine allocator round trips per transformer block.
    output_storage = torch.empty(
        (9, batch, seq, hidden), device=timestep.device, dtype=timestep.dtype
    )
    outs = tuple(output_storage.unbind(dim=0))
    _ltx2_ada_values9_kernel[(rows,)](
        timestep,
        scale_shift_table,
        *outs,
        rows,
        hidden,
        total_params,

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure the timestep projection outputs 9*D channels matching scale_shift_table.shape[1]
2. Fix hidden size mismatches between the projection layer and scale_shift_table
3. Add an assert timestep.shape[-1] == 9 * table.shape[1] at model init

Example fix

# before
vals = ltx2_ada_values9(table, t_6d)  # last dim 6*D
# after
assert t.shape[-1] == 9 * table.shape[1]
vals = ltx2_ada_values9(table, t)

assert timestep.shape[-1] == scale_shift_table.shape[0] * scale_shift_table.shape[1]

When it happens

Trigger: Passing a timestep projected with a 6-parameter AdaLN head (last dim 6*D), a hidden size that differs from the table's D, or raw unprojected timesteps whose last dim doesn't align.

Common situations: Model variants where the adaLN projection outputs a different number of parameter channels than the table expects; mismatched hidden sizes after config edits (e.g. resizing embeddings); wiring the wrong projection output into the fused path.

Related errors

• timestep must have shape [B, S, 9 * D]
• scale_shift_table must have shape [9, D]
• {name} must be a scalar tensor, got shape {tuple(scale.shape
• timestep must be a CUDA bfloat16 tensor
• timestep must be contiguous

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6f2f438b53768036. Report an issue: GitHub.