{"record":{"id":"67b35486829c4e7c","repo":"sgl-project/sglang","slug":"num-heads-num-epi-subtiles-must-be-divisible-by","errorCode":null,"errorMessage":"num_heads // num_epi_subtiles must be divisible by 4 (FMA unroll granularity)","messagePattern":"num_heads // num_epi_subtiles must be divisible by 4 \\(FMA unroll granularity\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/cutedsl_fp8_paged_mqa_logits.py","lineNumber":252,"sourceCode":"        self.num_heads = num_heads\n        self.head_dim = head_dim\n        self.next_n = next_n\n        self.N = next_n * num_heads\n        self.num_sms = num_sms\n        self.enable_pdl = enable_pdl\n        self.num_epi_subtiles = num_epi_subtiles\n        self.epi_dtype = epi_dtype\n        self.epi_bytes = 2 if epi_dtype == cutlass.Float16 else 4\n        # sW stage stride padded to 128-byte SMEM alignment for TMA bulk copy.\n        # Without padding, e.g. fp16 + N=32 gives 64B per stage, so stage 1\n        # at +64 would be misaligned (TMA requires 128-byte aligned SMEM dest).\n        w_stage_bytes = self.N * self.epi_bytes\n        self.w_stage_stride = ((w_stage_bytes + 127) // 128 * 128) // self.epi_bytes\n        self.output_dtype = output_dtype\n        if num_epi_subtiles > 1 and num_heads % num_epi_subtiles != 0:\n            raise ValueError(\"num_heads must be divisible by num_epi_subtiles\")\n        if (num_heads // num_epi_subtiles) % 4 != 0:\n            raise ValueError(\n                \"num_heads // num_epi_subtiles must be divisible by 4 (FMA unroll granularity)\"\n            )\n        self.num_groups = 2\n\n        self.num_math_threads = 256\n        self.num_specialized_threads = 128\n        self.threads_per_cta = 384\n        self.num_math_warps = 8\n        self.tma_warp_base = 8\n        self.umma_warp_base = 10\n\n        self.num_q_stages = 3  # 3 stages for Q pipelining across batch sequences\n\n        # TMEM: 512 columns total, each group needs N columns per UMMA stage\n        # max_umma_stages = 512 // (2 * N)\n        TMEM_COLS = 512\n        if max_umma_pipeline:\n            self.num_umma_stages = min(2, TMEM_COLS // (2 * self.N))","sourceCodeStart":234,"sourceCodeEnd":270,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/cutedsl_fp8_paged_mqa_logits.py#L234-L270","documentation":"The same kernel epilogue requires that after splitting heads into num_epi_subtiles subtiles, each subtile's head count is a multiple of 4 because the FMA math loop is unrolled by 4. __init__ raises ValueError when (num_heads // num_epi_subtiles) % 4 != 0.","triggerScenarios":"Constructing the fp8 paged MQA logits kernel where heads-per-subtile is not a multiple of 4 — e.g. num_heads=8, num_epi_subtiles=2 (4 ok) vs num_heads=12, num_epi_subtiles=2 (6 -> raises); head counts like 24 with subtile 2 (12) also fail.","commonSituations":"Models with head counts that are multiples of 4 per subtile requirement but not of 4*num_epi_subtiles overall; changing num_epi_subtiles for SMEM tuning and breaking the unroll granularity constraint.","solutions":["Ensure num_heads is divisible by 4 * num_epi_subtiles (e.g. num_heads=64 with subtile 4 gives 16 per subtile)","Reduce num_epi_subtiles until num_heads // num_epi_subtiles % 4 == 0","Fall back to num_epi_subtiles=1 if num_heads % 4 == 0 but no larger divisor works"],"exampleFix":"# before\nkernel = MqaLogitsKernel(num_heads=12, num_epi_subtiles=2)  # 12//2=6 -> ValueError\n# after\nkernel = MqaLogitsKernel(num_heads=12, num_epi_subtiles=1)  # 12//1=12 %4==0? no -> use heads divisible by 4\n# or num_heads=16, num_epi_subtiles=2 -> 8 % 4 == 0, ok","handlingStrategy":"validation","validationCode":"assert (num_heads // num_epi_subtiles) % 4 == 0, (\n    'num_heads // num_epi_subtiles must be divisible by 4 (FMA unroll granularity)')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat 4 * num_epi_subtiles as the effective head-divisibility requirement when tuning","Validate tiling params against num_heads in the model config loader, not at kernel build time"],"tags":["cutedsl","kernel-config","shape-validation","attention"],"backgroundTag":"shape-divisibility-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}