sgl-project/sglang · error · ValueError

Not a canonical UMMA_K Layout: Expected MN-size multiple of

Error message

Not a canonical UMMA_K Layout: Expected MN-size multiple of 8.

What it means

On the Major-K path, make_smem_desc_base requires the MN-mode extent (layout.shape[0]) to be a multiple of 8 so it can be divided into (8, 2) canonical atoms. Otherwise the descriptor bit-fields cannot represent the layout.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/mma_sm100_desc.py:275

        stride_00 = canonical_layout.stride[0][0]
        if layout_type is not LayoutType.SWIZZLE_NONE and stride_00 != 1:
            raise ValueError("Not a canonical UMMA_MN Layout: Expected stride failure.")
        stride_10 = canonical_layout.stride[1][0]
        if stride_10 != swizzle_atom_mn_size:
            raise ValueError("Not a canonical UMMA_MN Layout: Expected stride failure.")
        stride_01, stride_11 = (
            canonical_layout.stride[0][1],
            canonical_layout.stride[1][1],
        )
        if layout_type is LayoutType.SWIZZLE_NONE:
            stride_byte_offset, leading_byte_offset = stride_01, stride_11
        else:
            stride_byte_offset, leading_byte_offset = stride_11, stride_01
    else:
        if layout_type == LayoutType.SWIZZLE_128B_BASE32B:
            raise ValueError("SWIZZLE_128B_BASE32B is invalid for Major-K")
        if not cute.size(layout.shape[0]) % 8 == 0:
            raise ValueError(
                "Not a canonical UMMA_K Layout: Expected MN-size multiple of 8."
            )
        canonical_layout = cute.logical_divide(layout, (8, 2))
        if not cute.is_congruent(canonical_layout, ((1, 1), (1, 1))):
            raise ValueError("Not a canonical UMMA_K Layout: Expected profile failure.")
        stride_00 = canonical_layout.stride[0][0]
        if stride_00 != swizzle_atom_mn_size:
            raise ValueError("Not a canonical UMMA_K Layout: Expected stride failure.")
        stride_10 = canonical_layout.stride[1][0]
        if layout_type is not LayoutType.SWIZZLE_NONE and stride_10 != 1:
            raise ValueError("Not a canonical UMMA_K Layout: Expected stride failure.")
        stride_01 = canonical_layout.stride[0][1]
        stride_byte_offset, leading_byte_offset = stride_01, stride_10

    # ------------------------------------------------------------------ pack
    desc = 0
    # leading_byte_offset_  [16:30)
    desc |= (leading_byte_offset & 0x3FFF) << 16

View on GitHub (pinned to 0132848349)

Solutions

Pad or round the MN extent up to a multiple of 8 and predicate the unused columns/rows.
Choose standard tile sizes (64/128/256) that are inherently 8-aligned.
Verify which mode of your layout is mode 0 for Major.K before computing extents.

Defensive patterns

Strategy: validation

Validate before calling

if major is Major.K:
    assert cute.size(layout.shape[0]) % 8 == 0, "MN extent must be multiple of 8 for Major-K"

Prevention

Keep MN extents 8-aligned in K-major operand tiles.
Pad to alignment with predication instead of odd sizes.

When it happens

Trigger: Calling make_smem_desc_base with major=Major.K and an MN extent such as 4, 6, 12, or 20 (not divisible by 8).

Common situations: Non-standard head dims or tile N/M sizes when building K-major operand descriptors; shapes ported from SM90 kernels with different alignment requirements.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/649641fc23429a51. Report an issue: GitHub.