sgl-project/sglang · error · ValueError

n_q/n_k must be one of {VALID_N}, got n_q={n_q}, n_k={n_k}

Error message

n_q/n_k must be one of {VALID_N}, got n_q={n_q}, n_k={n_k}

What it means

The sub-block sparse attention router partitions each head into n_k key blocks and n_q query blocks, and only a fixed set VALID_N of block counts is supported (typically {1,2,4,8}). The constructor validates both n_k and n_q against VALID_N and raises ValueError otherwise.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/backends/subblock_sparse/router.py:184

class SubBlockRouter:
    """Builds ``q2k_block_index`` from sub-block-pooled Q/K.

    Args:
        n_k: key sub-blocks per 64-token block (1, 2, 4 or 8). 1 reproduces plain avg
            pooling; 4 is the quality/cost point the recall table above lands on.
        n_q: query sub-blocks, same values. Splitting Q *alone* (n_q>1 with n_k=1) is
            worse than not splitting; splitting both sides together is what the default
            does. Costs n_q times the score matrix, 0.5% of the denoise time.

    Structural block reservation (an attention sink, or forcing the diagonal j == i) was
    measured on 200 real H3 attention cells and is deliberately absent: at a fixed budget
    the diagonal changed relative L2 by 0.2% and the sink only helped in DiT layers 2-32,
    which did not survive to the pixels.
    """

    def __init__(self, n_k: int = 4, n_q: int = 4) -> None:
        if n_k not in VALID_N or n_q not in VALID_N:
            raise ValueError(
                f"n_q/n_k must be one of {VALID_N}, got n_q={n_q}, n_k={n_k}"
            )
        self.n_k, self.n_q = n_k, n_q

    @torch.no_grad()
    def scores(
        self, q: torch.Tensor, k: torch.Tensor, softmax_scale: float
    ) -> torch.Tensor:
        """``[B, S, H, D] -> [B, H, Gq, Gk]`` block scores (log-space, higher = keep).

        Two Triton kernels: pool, then GEMM + segmented log-sum-exp in registers, so the
        ``[B, H, Gq*n_q, Gk*n_k]`` intermediate never reaches memory.

        ``softmax_scale * log2(e)`` is folded into Q so the kernel can use the exp2/log2
        hardware instructions; it multiplies by ln 2 on the way out, so scores come back
        in natural-log units. Selection is a top-k and any monotone rescale leaves that
        alone, so the units only matter to a reader of the magnitudes.

View on GitHub (pinned to 0132848349)

Solutions

1. Set both n_q and n_k to values in VALID_N (check the module constant; usually powers of two like 1, 2, 4, 8).
2. Read VALID_N directly: `from sglang.multimodal_gen.runtime.layers.attention.backends.subblock_sparse.router import VALID_N; print(VALID_N)` and pick from it.
3. If you need an unsupported granularity, extend VALID_N in a fork after verifying the partition math in scores() supports it.

Example fix

# before
router = SubblockSparseRouter(n_k=3, n_q=6)  # not in VALID_N

# after
from ...subblock_sparse.router import VALID_N  # e.g. {1, 2, 4, 8}
router = SubblockSparseRouter(n_k=4, n_q=4)

from sglang.multimodal_gen.runtime.layers.attention.backends.subblock_sparse.router import VALID_N
assert n_q in VALID_N and n_k in VALID_N, f"n_q/n_k must be in {VALID_N}"
router = SubblockSparseRouter(n_k=n_k, n_q=n_q)

def is_valid_n(n: int) -> bool:
    return n in VALID_N

When it happens

Trigger: Constructing the router with block counts outside VALID_N, e.g. SubblockSparseRouter(n_k=3, n_q=4), n_k=6, or 0; even one invalid value triggers the raise.

Common situations: Tuning sparsity hyperparameters with values like 3 or 6 that don't match power-of-two block grids; copying configs from papers/repo forks with different VALID_N sets; dividing head_dim by a granularity not supported after a version change.

Related errors

• Invalid attention metadata values.Sparsity should be in [0,
• f"Unknown feature map: {feature_map}"
• f"Unsupported patch_size type: {type(patch_size)}"
• memory_size must be positive
• consumer_count must be positive

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f840d55b6dfee740. Report an issue: GitHub.