sgl-project/sglang · error · ValueError

Invalid select_mode: {select_mode}. Choose 'topk' or 'thresh

Error message

Invalid select_mode: {select_mode}. Choose 'topk' or 'threshold'.

What it means

moba_attn_varlen only supports two gate selection modes: 'topk' and 'threshold'. Any other select_mode string falls through to the final else and raises ValueError.

Source

Thrown at python/sglang/multimodal_gen/csrc/attn/vmoba_attn/vmoba/vmoba.py:772

            )
        elif threshold_type == "block":
            gate_mask = _select_threshold_block(
                gate, valid_gate_mask, gate_self_chunk_mask, simsum_threshold
            )
        elif threshold_type == "overall":
            gate_mask = _select_threshold_overall(
                gate, valid_gate_mask, gate_self_chunk_mask, simsum_threshold
            )
        elif threshold_type == "head_global":
            gate_mask = _select_threshold_head_global(
                gate, valid_gate_mask, gate_self_chunk_mask, simsum_threshold
            )
        else:
            raise ValueError(
                f"Invalid threshold_type: {threshold_type}. Choose 'query_head', 'block', or 'overall'."
            )
    else:
        raise ValueError(
            f"Invalid select_mode: {select_mode}. Choose 'topk' or 'threshold'."
        )

    # eliminate self_chunk in MoBA branch
    gate_mask = gate_mask & ~gate_self_chunk_mask
    # if gate_mask is all false, perform flash_attn instead
    if gate_mask.sum() == 0:
        return flash_attn_varlen_func(
            q, k, v, cu_seqlens, cu_seqlens, max_seqlen, max_seqlen, causal=False
        )

    # Determine which query positions are selected.
    # nonzero_indices has shape [N, 3] where each row is [chunk_index, head_index, seq_index].
    moba_q_indices = gate_mask.reshape(gate_mask.shape[0], -1).nonzero(as_tuple=True)[
        -1
    ]  # [(h s k)]
    moba_q_sh_indices = (moba_q_indices % seqlen) * num_head + (
        moba_q_indices // seqlen

View on GitHub (pinned to 0132848349)

Solutions

Set select_mode to exactly 'topk' or 'threshold' (case-sensitive)
If the value comes from a config, validate/normalize it before the call and ensure it is not None

Example fix

# before
moba_attn_varlen(..., select_mode="top_k")
# after
moba_attn_varlen(..., select_mode="topk")

Defensive patterns

Strategy: validation

Validate before calling

VALID_SELECT_MODES = {"topk", "threshold"}
if select_mode not in VALID_SELECT_MODES:
    raise ValueError(f"select_mode must be one of {VALID_SELECT_MODES}, got {select_mode!r}")

Type guard

def is_valid_select_mode(m: str) -> bool:
    return m in {"topk", "threshold"}

Try / catch

try:
    moba_attn_varlen(..., select_mode=select_mode)
except ValueError as e:
    if "Invalid select_mode" in str(e):
        select_mode = "topk"
    else:
        raise

Prevention

Validate select_mode at config parse time with an allowlist
Default the field so it is never None
Add a smoke test that each documented mode string runs

When it happens

Trigger: Calling moba_attn_varlen(..., select_mode=<not 'topk' or 'threshold'>), e.g. 'top_k', 'Threshold', 'score'.

Common situations: Typo or casing mistake in the mode string; configs from a different library version that renamed modes; defaulting select_mode from an unset config variable (None).

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/703f481b99a0c130. Report an issue: GitHub.