sgl-project/sglang · error · ValueError

attn_res: nvb must be in [1, {_MAX_BANK_ROWS}], got {nvb}

Error message

attn_res: nvb must be in [1, {_MAX_BANK_ROWS}], got {nvb}

What it means

_tuning selects (chunk_rows, occupancy, consumer_regs) launch parameters from a table indexed by nvb (number of value-bank rows), which only has entries for nvb in [1, _MAX_BANK_ROWS]. An out-of-range nvb has no tuned config, so it is rejected before indexing the table.

Source

Thrown at python/sglang/kernels/ops/kimi_k3/attn_res.py:92

_TMA_BEST_CONFIG: dict[int, tuple[int, int, int]] = {
    1: (2, 2, 0),
    2: (4, 1, 200),
    3: (4, 1, 200),
    4: (5, 1, 200),
    5: (3, 1, 200),
    6: (4, 1, 200),
    7: (4, 1, 200),
    8: (5, 1, 200),
}


def _tuning(nvb: int, num_tokens: int) -> tuple[int, int, int]:
    """(chunk_rows, occupancy, consumer_regs) for this aggregation point.

    Shared by all three entry points below: they run the same kernel template
    and differ only in the collective fused onto it."""
    if not 1 <= nvb <= _MAX_BANK_ROWS:
        raise ValueError(f"attn_res: nvb must be in [1, {_MAX_BANK_ROWS}], got {nvb}")
    best = _TMA_BEST_CONFIG[nvb]
    if best[1] > 1 and num_tokens < 128:
        # occupancy=2 only pays off once there are enough tokens to fill both
        # CTAs per SM; below that its tighter register budget just costs ~10%.
        best = (4, 1, 200)
    return best


_COMM_MAP: dict[int, Communicator] = {}


def register_comm(comm: Communicator) -> None:
    # One communicator per world_size per process -- see the note in
    # kimi_k3/all_reduce.py::register_comm. The ops key only on world_size, so an
    # overwrite here would hand the old group's callers the new group's peer
    # pointers.
    prev = _COMM_MAP.get(comm.world_size)
    assert prev is None or prev is comm, (

View on GitHub (pinned to 0132848349)

Solutions

1. Clamp or validate nvb to [1, _MAX_BANK_ROWS] at the call site and skip/split the work when it exceeds the max
2. If nvb legitimately exceeds _MAX_BANK_ROWS, split the aggregation into multiple passes with nvb within range
3. Check how nvb is computed (bank rows) for off-by-one or empty-bank (0) cases

Example fix

# before
attn_res_fused_tma(x, ..., nvb=nvb)
# after
nvb = max(1, min(nvb, _MAX_BANK_ROWS))
attn_res_fused_tma(x, ..., nvb=nvb)

assert 1 <= nvb <= _MAX_BANK_ROWS, f'nvb {nvb} out of range'

def valid_nvb(nvb: int) -> bool:
    return isinstance(nvb, int) and 1 <= nvb <= _MAX_BANK_ROWS

When it happens

Trigger: Calling attn_res_fused_tma / attn_res_fused_direct_ag / attn_res_fused_pull_rs with an nvb (bank row count) of 0 or greater than _MAX_BANK_ROWS, usually derived from a bank/row dimension of the attention-residual tensors.

Common situations: Changing the Kimi K3 bank layout or head configuration so the derived nvb exceeds the tuning table's maximum; passing 0 for an empty bank instead of skipping the call.

Related errors

• attn_sink requires topk_length to be provided as well
• missing value for {a} (expected e.g. `{a} 2,4`)
• combined_history=True requires direction=0 (bidi)
• kv-canary: launch_canary_plan_kernels requires full_to_swa_i
• kv-canary: launch_canary_plan_kernels verify_capacity={verif

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/df0e385233acd815. Report an issue: GitHub.