sgl-project/sglang · error · ValueError

Invalid threshold_type: {threshold_type}. Choose 'query_head

Error message

Invalid threshold_type: {threshold_type}. Choose 'query_head', 'block', or 'overall'.

What it means

Inside the select_mode == 'threshold' branch, moba_attn_varlen dispatches on threshold_type to a specific thresholding helper (_select_threshold_*). Values other than the supported set (including 'query_head'/'block'/'overall' style names not valid here, e.g. anything not handled above 'head_global') raise ValueError.

Source

Thrown at python/sglang/multimodal_gen/csrc/attn/vmoba_attn/vmoba/vmoba.py:768

        valid_gate_mask = gate != -float("inf")  # (num_chunk, num_head, seqlen)
        if threshold_type == "query_head":
            gate_mask = _select_threshold_query_head(
                gate, valid_gate_mask, gate_self_chunk_mask, simsum_threshold
            )
        elif threshold_type == "block":
            gate_mask = _select_threshold_block(
                gate, valid_gate_mask, gate_self_chunk_mask, simsum_threshold
            )
        elif threshold_type == "overall":
            gate_mask = _select_threshold_overall(
                gate, valid_gate_mask, gate_self_chunk_mask, simsum_threshold
            )
        elif threshold_type == "head_global":
            gate_mask = _select_threshold_head_global(
                gate, valid_gate_mask, gate_self_chunk_mask, simsum_threshold
            )
        else:
            raise ValueError(
                f"Invalid threshold_type: {threshold_type}. Choose 'query_head', 'block', or 'overall'."
            )
    else:
        raise ValueError(
            f"Invalid select_mode: {select_mode}. Choose 'topk' or 'threshold'."
        )

    # eliminate self_chunk in MoBA branch
    gate_mask = gate_mask & ~gate_self_chunk_mask
    # if gate_mask is all false, perform flash_attn instead
    if gate_mask.sum() == 0:
        return flash_attn_varlen_func(
            q, k, v, cu_seqlens, cu_seqlens, max_seqlen, max_seqlen, causal=False
        )

    # Determine which query positions are selected.
    # nonzero_indices has shape [N, 3] where each row is [chunk_index, head_index, seq_index].
    moba_q_indices = gate_mask.reshape(gate_mask.shape[0], -1).nonzero(as_tuple=True)[

View on GitHub (pinned to 0132848349)

Solutions

Use a threshold_type handled by the threshold branch (e.g. 'head_global' or the other _select_threshold_* names defined in the file)
If you intended top-k selection, use select_mode='topk' with 'query_head'/'block'/'overall'
Print/grep the valid threshold_type literals near vmoba.py:740-770 to confirm supported names for your version

Example fix

# before
moba_attn_varlen(..., select_mode="threshold", threshold_type="query_head")
# after
moba_attn_varlen(..., select_mode="threshold", threshold_type="head_global")

Defensive patterns

Strategy: validation

Validate before calling

# confirm against the literals handled in vmoba.py's threshold branch
VALID_THRESHOLD_TYPES = {"head_global"}  # extend per your vmoba.py version
if select_mode == "threshold":
    assert threshold_type in VALID_THRESHOLD_TYPES, threshold_type

Type guard

def is_valid_threshold_type(t: str) -> bool:
    return t in {"head_global"}  # keep in sync with _select_threshold_* dispatch

Try / catch

try:
    moba_attn_varlen(..., select_mode="threshold", threshold_type=threshold_type)
except ValueError as e:
    if "Invalid threshold_type" in str(e):
        threshold_type = "head_global"
    else:
        raise

Prevention

Grep vmoba.py for `_select_threshold_` helpers to enumerate valid names each version
Fail fast on unknown config strings during model init
Keep mode/threshold_type pairs together in a single config dataclass

When it happens

Trigger: Calling moba_attn_varlen(..., select_mode='threshold', threshold_type=<unhandled value>) — any string not matched by the if/elif chain ending at 'head_global'.

Common situations: Passing a topk-style value such as 'overall' or 'block' when select_mode='threshold' if those aren't handled in the chain; typo'd config keys; configs migrated between modes.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a49d6de803f2c9a0. Report an issue: GitHub.