{"record":{"id":"38130c5b74570161","repo":"sgl-project/sglang","slug":"num-heads-must-be-divisible-by-num-epi-subtiles","errorCode":null,"errorMessage":"num_heads must be divisible by num_epi_subtiles","messagePattern":"num_heads must be divisible by num_epi_subtiles","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/cutedsl_fp8_paged_mqa_logits.py","lineNumber":250,"sourceCode":"        self.smem_subpartition_opt = smem_subpartition_opt\n        self.max_w_in_reg = max_w_in_reg\n        self.num_heads = num_heads\n        self.head_dim = head_dim\n        self.next_n = next_n\n        self.N = next_n * num_heads\n        self.num_sms = num_sms\n        self.enable_pdl = enable_pdl\n        self.num_epi_subtiles = num_epi_subtiles\n        self.epi_dtype = epi_dtype\n        self.epi_bytes = 2 if epi_dtype == cutlass.Float16 else 4\n        # sW stage stride padded to 128-byte SMEM alignment for TMA bulk copy.\n        # Without padding, e.g. fp16 + N=32 gives 64B per stage, so stage 1\n        # at +64 would be misaligned (TMA requires 128-byte aligned SMEM dest).\n        w_stage_bytes = self.N * self.epi_bytes\n        self.w_stage_stride = ((w_stage_bytes + 127) // 128 * 128) // self.epi_bytes\n        self.output_dtype = output_dtype\n        if num_epi_subtiles > 1 and num_heads % num_epi_subtiles != 0:\n            raise ValueError(\"num_heads must be divisible by num_epi_subtiles\")\n        if (num_heads // num_epi_subtiles) % 4 != 0:\n            raise ValueError(\n                \"num_heads // num_epi_subtiles must be divisible by 4 (FMA unroll granularity)\"\n            )\n        self.num_groups = 2\n\n        self.num_math_threads = 256\n        self.num_specialized_threads = 128\n        self.threads_per_cta = 384\n        self.num_math_warps = 8\n        self.tma_warp_base = 8\n        self.umma_warp_base = 10\n\n        self.num_q_stages = 3  # 3 stages for Q pipelining across batch sequences\n\n        # TMEM: 512 columns total, each group needs N columns per UMMA stage\n        # max_umma_stages = 512 // (2 * N)\n        TMEM_COLS = 512","sourceCodeStart":232,"sourceCodeEnd":268,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/cutedsl_fp8_paged_mqa_logits.py#L232-L268","documentation":"The fp8 paged MQA logits CuteDSL kernel's TMA-based epilogue splits the head dimension into num_epi_subtiles for SMEM staging, which requires the split to be exact. The kernel __init__ raises ValueError when num_epi_subtiles > 1 and num_heads % num_epi_subtiles != 0.","triggerScenarios":"Instantiating the MQA logits kernel class with num_epi_subtiles > 1 while num_heads is not a multiple of it — e.g. num_heads=32, num_epi_subtiles=6; or a caller computing num_epi_subtiles from SMEM budget without aligning it to the model's head count.","commonSituations":"Tuning a new GQA/MQA model config (odd head counts like 48 with an epi subtile of 5) where the epilogue tiling was tuned for a different model; passing num_epi_subtiles derived from tile-size heuristics rather than a divisor of num_heads.","solutions":["Pick num_epi_subtiles as a power-of-2 divisor of num_heads (1, 2, 4, 8...)","Use num_epi_subtiles=1 to bypass the constraint (with possible perf cost)","Add an assertion at the caller: assert num_heads % num_epi_subtiles == 0 before constructing the kernel"],"exampleFix":"# before\nkernel = MqaLogitsKernel(num_heads=48, num_epi_subtiles=5)  # ValueError\n# after\nkernel = MqaLogitsKernel(num_heads=48, num_epi_subtiles=4)  # 48 % 4 == 0","handlingStrategy":"validation","validationCode":"assert num_epi_subtiles == 1 or num_heads % num_epi_subtiles == 0, (\n    f'num_heads ({num_heads}) must be divisible by num_epi_subtiles ({num_epi_subtiles})')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Derive num_epi_subtiles only from power-of-2 divisors of num_heads","Add a config sanity check in model load that validates all kernel tiling constraints at once"],"tags":["cutedsl","kernel-config","shape-validation","attention"],"backgroundTag":"shape-divisibility-validation","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}