sgl-project/sglang · error · ValueError

num_heads // num_epi_subtiles must be divisible by 4 (FMA un

Error message

num_heads // num_epi_subtiles must be divisible by 4 (FMA unroll granularity)

What it means

The same kernel epilogue requires that after splitting heads into num_epi_subtiles subtiles, each subtile's head count is a multiple of 4 because the FMA math loop is unrolled by 4. __init__ raises ValueError when (num_heads // num_epi_subtiles) % 4 != 0.

Source

Thrown at python/sglang/kernels/ops/attention/cutedsl_fp8_paged_mqa_logits.py:252

        self.num_heads = num_heads
        self.head_dim = head_dim
        self.next_n = next_n
        self.N = next_n * num_heads
        self.num_sms = num_sms
        self.enable_pdl = enable_pdl
        self.num_epi_subtiles = num_epi_subtiles
        self.epi_dtype = epi_dtype
        self.epi_bytes = 2 if epi_dtype == cutlass.Float16 else 4
        # sW stage stride padded to 128-byte SMEM alignment for TMA bulk copy.
        # Without padding, e.g. fp16 + N=32 gives 64B per stage, so stage 1
        # at +64 would be misaligned (TMA requires 128-byte aligned SMEM dest).
        w_stage_bytes = self.N * self.epi_bytes
        self.w_stage_stride = ((w_stage_bytes + 127) // 128 * 128) // self.epi_bytes
        self.output_dtype = output_dtype
        if num_epi_subtiles > 1 and num_heads % num_epi_subtiles != 0:
            raise ValueError("num_heads must be divisible by num_epi_subtiles")
        if (num_heads // num_epi_subtiles) % 4 != 0:
            raise ValueError(
                "num_heads // num_epi_subtiles must be divisible by 4 (FMA unroll granularity)"
            )
        self.num_groups = 2

        self.num_math_threads = 256
        self.num_specialized_threads = 128
        self.threads_per_cta = 384
        self.num_math_warps = 8
        self.tma_warp_base = 8
        self.umma_warp_base = 10

        self.num_q_stages = 3  # 3 stages for Q pipelining across batch sequences

        # TMEM: 512 columns total, each group needs N columns per UMMA stage
        # max_umma_stages = 512 // (2 * N)
        TMEM_COLS = 512
        if max_umma_pipeline:
            self.num_umma_stages = min(2, TMEM_COLS // (2 * self.N))

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure num_heads is divisible by 4 * num_epi_subtiles (e.g. num_heads=64 with subtile 4 gives 16 per subtile)
2. Reduce num_epi_subtiles until num_heads // num_epi_subtiles % 4 == 0
3. Fall back to num_epi_subtiles=1 if num_heads % 4 == 0 but no larger divisor works

Example fix

# before
kernel = MqaLogitsKernel(num_heads=12, num_epi_subtiles=2)  # 12//2=6 -> ValueError
# after
kernel = MqaLogitsKernel(num_heads=12, num_epi_subtiles=1)  # 12//1=12 %4==0? no -> use heads divisible by 4
# or num_heads=16, num_epi_subtiles=2 -> 8 % 4 == 0, ok

assert (num_heads // num_epi_subtiles) % 4 == 0, (
    'num_heads // num_epi_subtiles must be divisible by 4 (FMA unroll granularity)')

When it happens

Trigger: Constructing the fp8 paged MQA logits kernel where heads-per-subtile is not a multiple of 4 — e.g. num_heads=8, num_epi_subtiles=2 (4 ok) vs num_heads=12, num_epi_subtiles=2 (6 -> raises); head counts like 24 with subtile 2 (12) also fail.

Common situations: Models with head counts that are multiples of 4 per subtile requirement but not of 4*num_epi_subtiles overall; changing num_epi_subtiles for SMEM tuning and breaking the unroll granularity constraint.

Related errors

• num_heads must be divisible by num_epi_subtiles
• Unexpected initial_state_source shape: {initial_state_source
• Unexpected A_log shape: {A_log.shape}; expected numel={HV}
• Unexpected dt_bias shape: {dt_bias.shape}; expected numel={H
• Unexpected a shape for varlen: {a.shape}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/67b35486829c4e7c. Report an issue: GitHub.