{"record":{"id":"3303e19780a637e2","repo":"sgl-project/sglang","slug":"tiny-k-gemm-no-valid-split-n-for-n-n-k-k","errorCode":null,"errorMessage":"tiny_k_gemm: no valid split_n for N={n}, K={k}","messagePattern":"tiny_k_gemm: no valid split_n for N=(.+?), K=(.+?)","errorType":"error_code","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/gemm/tiny_gemm.py","lineNumber":107,"sourceCode":"        assert out_dtype is None or out_dtype == out.dtype\n    if split_n is None:\n        split_n = _default_split_n(n, k, max_m, x.device)\n    module = _jit_tiny_gemm_module(n, k, max_m, split_n, out.dtype)\n    module.run(x, w, out)\n    return out\n\n\ndef _default_k_split_n(n: int, k: int) -> int:\n    \"\"\"Smallest divisor of n whose n / split_n blocks fit one wave, with\n    split_n * K-lanes whole-warp aligned and within the block-size limit.\"\"\"\n    lanes = k // 8  # fixed 16-byte vectors in the K variant\n    candidates = [\n        d\n        for d in range(1, n + 1)\n        if n % d == 0 and d * lanes % 32 == 0 and d * lanes <= 1024\n    ]\n    if not candidates:\n        raise RuntimeError(f\"tiny_k_gemm: no valid split_n for N={n}, K={k}\")\n    sm_count = torch.cuda.get_device_properties(0).multi_processor_count\n    for d in candidates:\n        if n // d <= sm_count:\n            return d\n    return candidates[-1]\n\n\ndef tiny_k_gemm_bf16(\n    x: torch.Tensor,\n    w: torch.Tensor,\n    out: Optional[torch.Tensor] = None,\n    *,\n    out_dtype: Optional[torch.dtype] = None,\n    split_n: Optional[int] = None,\n    max_m: int = _MAX_M_DEFAULT,\n) -> torch.Tensor:\n    \"\"\"Small-K / large-N variant: K / 8 lanes of one warp reduce the K\n    dimension for one output column; each block covers split_n columns and the","sourceCodeStart":89,"sourceCodeEnd":125,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/gemm/tiny_gemm.py#L89-L125","documentation":"tiny_k_gemm_bf16 picks a split_n divisor d of N such that d*lanes is a multiple of 32 and d*lanes <= 1024. If no divisor of N satisfies those constraints, the candidate list is empty and the kernel refuses to launch because no valid work decomposition exists.","triggerScenarios":"Calling tiny_k_gemm_bf16 with an N that has no divisor d meeting (n % d == 0 and d*lanes % 32 == 0 and d*lanes <= 1024) — typically odd/prime N combined with a lanes value whose multiples never hit a multiple of 32 within the 1024 cap (e.g. lanes=3 with odd N).","commonSituations":"Non-power-of-two output dimensions in small GEMMs, unusual head sizes or intermediate dims, or a lanes parameter changed during tuning that breaks divisibility.","solutions":["Check N and lanes; if N is odd or prime relative to 32/lanes divisibility, pad the output dimension N up to the next multiple that has a valid divisor (e.g. next multiple of 32/lanes)","Adjust the lanes argument so d*lanes can be a multiple of 32 with d dividing N","Fall back to a standard torch.matmul path for this shape if padding is not acceptable"],"exampleFix":"# before\ny = tiny_k_gemm_bf16(x, w, n=1150, lanes=1)  # 1150 has no valid split_n\n# after\npad = (-n) % 32\ny = tiny_k_gemm_bf16(x, w, n=n+pad, lanes=1)[:n]","handlingStrategy":"validation","validationCode":"lanes = 1  # given\nvalid = any(n % d == 0 and d*lanes % 32 == 0 and d*lanes <= 1024 for d in range(1, n+1))\nif not valid:\n    n_padded = n + (-n) % 32  # pad then slice output","typeGuard":"def tiny_k_gemm_shape_ok(n: int, lanes: int) -> bool:\n    return any(n % d == 0 and d*lanes % 32 == 0 and d*lanes <= 1024 for d in range(1, n+1))","tryCatchPattern":null,"preventionTips":["Keep N a multiple of 32/lanes when targeting tiny_k_gemm_bf16","Pre-check divisor candidates before launching","Wrap model-specific tiny GEMMs in a shape-compat helper that falls back to torch.matmul"],"tags":["cuda","gemm","shape-validation","tiny-kernel"],"backgroundTag":"invalid-shape-argument","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}