{"record":{"id":"6f2f438b53768036","repo":"sgl-project/sglang","slug":"timestep-last-dim-must-equal-9-hidden","errorCode":null,"errorMessage":"timestep last dim must equal 9 * hidden","messagePattern":"timestep last dim must equal 9 \\* hidden","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/diffusion/modulate/ltx2_ada_values_triton.py","lineNumber":162,"sourceCode":"    if not timestep.is_cuda or timestep.dtype != torch.bfloat16:\n        raise ValueError(\"timestep must be a CUDA bfloat16 tensor\")\n    if not timestep.is_contiguous():\n        raise ValueError(\"timestep must be contiguous\")\n    if scale_shift_table.ndim != 2 or scale_shift_table.shape[0] != 9:\n        raise ValueError(\"scale_shift_table must have shape [9, D]\")\n    if (\n        not scale_shift_table.is_cuda\n        or scale_shift_table.dtype not in (torch.bfloat16, torch.float32)\n        or scale_shift_table.stride(-1) != 1\n    ):\n        raise ValueError(\n            \"scale_shift_table must be CUDA, bf16/fp32, last-dim contiguous\"\n        )\n\n    total_params = int(scale_shift_table.shape[0])\n    hidden = int(scale_shift_table.shape[1])\n    if hidden <= 0 or timestep.shape[-1] != total_params * hidden:\n        raise ValueError(\"timestep last dim must equal 9 * hidden\")\n    if hidden % 256 != 0 or hidden > 8192:\n        raise ValueError(\"hidden size is outside the supported LTX2 fast-path range\")\n\n    batch, seq, _ = timestep.shape\n    rows = int(batch * seq)\n    # Each returned output is a disjoint, contiguous view, so one allocation\n    # avoids nine allocator round trips per transformer block.\n    output_storage = torch.empty(\n        (9, batch, seq, hidden), device=timestep.device, dtype=timestep.dtype\n    )\n    outs = tuple(output_storage.unbind(dim=0))\n    _ltx2_ada_values9_kernel[(rows,)](\n        timestep,\n        scale_shift_table,\n        *outs,\n        rows,\n        hidden,\n        total_params,","sourceCodeStart":144,"sourceCodeEnd":180,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/diffusion/modulate/ltx2_ada_values_triton.py#L144-L180","documentation":"The timestep embedding's last dimension must be exactly total_params * hidden = 9 * D, matching the scale_shift_table's [9, D] shape. A mismatch (e.g. 6*D from a standard AdaLN projection, or a different hidden size) raises this ValueError.","triggerScenarios":"Passing a timestep projected with a 6-parameter AdaLN head (last dim 6*D), a hidden size that differs from the table's D, or raw unprojected timesteps whose last dim doesn't align.","commonSituations":"Model variants where the adaLN projection outputs a different number of parameter channels than the table expects; mismatched hidden sizes after config edits (e.g. resizing embeddings); wiring the wrong projection output into the fused path.","solutions":["Ensure the timestep projection outputs 9*D channels matching scale_shift_table.shape[1]","Fix hidden size mismatches between the projection layer and scale_shift_table","Add an assert timestep.shape[-1] == 9 * table.shape[1] at model init"],"exampleFix":"# before\nvals = ltx2_ada_values9(table, t_6d)  # last dim 6*D\n# after\nassert t.shape[-1] == 9 * table.shape[1]\nvals = ltx2_ada_values9(table, t)","handlingStrategy":"validation","validationCode":"assert timestep.shape[-1] == scale_shift_table.shape[0] * scale_shift_table.shape[1]","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Match adaLN projection output channels to the table (9*D)","Add shape consistency asserts at model init"],"tags":["shape","ltx2","adaln","dimension-mismatch"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}