sgl-project/sglang · error · ValueError

scale_shift_table must be CUDA, bf16/fp32, last-dim contiguo

Error message

scale_shift_table must be CUDA, bf16/fp32, last-dim contiguous

What it means

Beyond shape, scale_shift_table must be a CUDA tensor in bfloat16 or float32 with unit stride on the last dim. CPU tensors, other dtypes, or last-dim non-contiguous tables trigger this error.

Source

Thrown at python/sglang/kernels/ops/diffusion/modulate/ltx2_ada_values_triton.py:155

def ltx2_ada_values9(
    scale_shift_table: torch.Tensor,
    timestep: torch.Tensor,
) -> tuple[torch.Tensor, ...]:
    if timestep.ndim != 3:
        raise ValueError("timestep must have shape [B, S, 9 * D]")
    if not timestep.is_cuda or timestep.dtype != torch.bfloat16:
        raise ValueError("timestep must be a CUDA bfloat16 tensor")
    if not timestep.is_contiguous():
        raise ValueError("timestep must be contiguous")
    if scale_shift_table.ndim != 2 or scale_shift_table.shape[0] != 9:
        raise ValueError("scale_shift_table must have shape [9, D]")
    if (
        not scale_shift_table.is_cuda
        or scale_shift_table.dtype not in (torch.bfloat16, torch.float32)
        or scale_shift_table.stride(-1) != 1
    ):
        raise ValueError(
            "scale_shift_table must be CUDA, bf16/fp32, last-dim contiguous"
        )

    total_params = int(scale_shift_table.shape[0])
    hidden = int(scale_shift_table.shape[1])
    if hidden <= 0 or timestep.shape[-1] != total_params * hidden:
        raise ValueError("timestep last dim must equal 9 * hidden")
    if hidden % 256 != 0 or hidden > 8192:
        raise ValueError("hidden size is outside the supported LTX2 fast-path range")

    batch, seq, _ = timestep.shape
    rows = int(batch * seq)
    # Each returned output is a disjoint, contiguous view, so one allocation
    # avoids nine allocator round trips per transformer block.
    output_storage = torch.empty(
        (9, batch, seq, hidden), device=timestep.device, dtype=timestep.dtype
    )
    outs = tuple(output_storage.unbind(dim=0))

View on GitHub (pinned to 0132848349)

Solutions

1. table = table.to(device='cuda') and cast to bf16 or fp32 (match timestep precision convention)
2. If transposed, apply table.t().contiguous() to get [9, D] with stride-1 rows
3. Add a startup assert on table.is_cuda and dtype

Example fix

# before
vals = ltx2_ada_values9(table_cpu_fp16, t)
# after
table = table_cpu_fp16.to('cuda').to(torch.bfloat16)
vals = ltx2_ada_values9(table, t)

assert scale_shift_table.is_cuda and scale_shift_table.dtype in (torch.bfloat16, torch.float32) and scale_shift_table.stride(-1) == 1
scale_shift_table = scale_shift_table.to('cuda', torch.bfloat16).contiguous()

def table_props_ok(t: torch.Tensor) -> bool:
    return t.is_cuda and t.dtype in (torch.bfloat16, torch.float32) and t.stride(-1) == 1

When it happens

Trigger: Table on CPU (common right after checkpoint load without .to('cuda')), an fp16 table, or a table produced by a transpose making the last dim strided.

Common situations: Forgetting to move model parameters to GPU after loading; keeping the table in fp16 while timestep is bf16; transposed layouts from external checkpoints.

Related errors

• q, k, and v must have the same device and dtype
• timestep must be a CUDA bfloat16 tensor
• timestep must be contiguous
• QKV tensors must be CUDA bfloat16 tensors
• Reserved serve backend names cannot be used: {names}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/065dc0246fbc1af2. Report an issue: GitHub.