{"record":{"id":"df0e385233acd815","repo":"sgl-project/sglang","slug":"attn-res-nvb-must-be-in-1-max-bank-rows-go","errorCode":null,"errorMessage":"attn_res: nvb must be in [1, {_MAX_BANK_ROWS}], got {nvb}","messagePattern":"attn_res: nvb must be in \\[1, (.+?)\\], got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kimi_k3/attn_res.py","lineNumber":92,"sourceCode":"_TMA_BEST_CONFIG: dict[int, tuple[int, int, int]] = {\n    1: (2, 2, 0),\n    2: (4, 1, 200),\n    3: (4, 1, 200),\n    4: (5, 1, 200),\n    5: (3, 1, 200),\n    6: (4, 1, 200),\n    7: (4, 1, 200),\n    8: (5, 1, 200),\n}\n\n\ndef _tuning(nvb: int, num_tokens: int) -> tuple[int, int, int]:\n    \"\"\"(chunk_rows, occupancy, consumer_regs) for this aggregation point.\n\n    Shared by all three entry points below: they run the same kernel template\n    and differ only in the collective fused onto it.\"\"\"\n    if not 1 <= nvb <= _MAX_BANK_ROWS:\n        raise ValueError(f\"attn_res: nvb must be in [1, {_MAX_BANK_ROWS}], got {nvb}\")\n    best = _TMA_BEST_CONFIG[nvb]\n    if best[1] > 1 and num_tokens < 128:\n        # occupancy=2 only pays off once there are enough tokens to fill both\n        # CTAs per SM; below that its tighter register budget just costs ~10%.\n        best = (4, 1, 200)\n    return best\n\n\n_COMM_MAP: dict[int, Communicator] = {}\n\n\ndef register_comm(comm: Communicator) -> None:\n    # One communicator per world_size per process -- see the note in\n    # kimi_k3/all_reduce.py::register_comm. The ops key only on world_size, so an\n    # overwrite here would hand the old group's callers the new group's peer\n    # pointers.\n    prev = _COMM_MAP.get(comm.world_size)\n    assert prev is None or prev is comm, (","sourceCodeStart":74,"sourceCodeEnd":110,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kimi_k3/attn_res.py#L74-L110","documentation":"_tuning selects (chunk_rows, occupancy, consumer_regs) launch parameters from a table indexed by nvb (number of value-bank rows), which only has entries for nvb in [1, _MAX_BANK_ROWS]. An out-of-range nvb has no tuned config, so it is rejected before indexing the table.","triggerScenarios":"Calling attn_res_fused_tma / attn_res_fused_direct_ag / attn_res_fused_pull_rs with an nvb (bank row count) of 0 or greater than _MAX_BANK_ROWS, usually derived from a bank/row dimension of the attention-residual tensors.","commonSituations":"Changing the Kimi K3 bank layout or head configuration so the derived nvb exceeds the tuning table's maximum; passing 0 for an empty bank instead of skipping the call.","solutions":["Clamp or validate nvb to [1, _MAX_BANK_ROWS] at the call site and skip/split the work when it exceeds the max","If nvb legitimately exceeds _MAX_BANK_ROWS, split the aggregation into multiple passes with nvb within range","Check how nvb is computed (bank rows) for off-by-one or empty-bank (0) cases"],"exampleFix":"# before\nattn_res_fused_tma(x, ..., nvb=nvb)\n# after\nnvb = max(1, min(nvb, _MAX_BANK_ROWS))\nattn_res_fused_tma(x, ..., nvb=nvb)","handlingStrategy":"validation","validationCode":"assert 1 <= nvb <= _MAX_BANK_ROWS, f'nvb {nvb} out of range'","typeGuard":"def valid_nvb(nvb: int) -> bool:\n    return isinstance(nvb, int) and 1 <= nvb <= _MAX_BANK_ROWS","tryCatchPattern":null,"preventionTips":["Derive nvb from validated bank-layout constants","Add a range assert in tuning/benchmark scripts","Split oversized bank-row work instead of forcing one call"],"tags":["kimi-k3","tuning","argument-validation","range-check"],"backgroundTag":"argument-out-of-range","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}