What causes "SM120 relative bias requires tile_mn=(64, 128)" in sgl-project/sglang?

Pass tile_mn=None when has_bias=True and let select_config choose (64, 128)

sgl-project/sglang · error · ValueError

SM120 relative bias requires tile_mn=(64, 128)

Error message

SM120 relative bias requires tile_mn=(64, 128)

What it means

When relative bias is enabled on the SM120 FlashAttention path, the only supported tile is (64, 128); the kernel hardcodes that tile after selecting it. If the caller explicitly passes a different tile_mn, select_config rejects it.

Source

Thrown at python/sglang/kernels/ops/attention/fa4_sm120/runtime.py:632

        seqlen_q: Optional[int],
        seqlen_k: Optional[int],
        num_head_kv: int,
        qhead_per_kvhead: int,
        is_causal: bool,
        is_local: bool,
        window_size_left: Optional[int],
        window_size_right: Optional[int],
        pack_gqa: bool,
        paged_kv: bool,
    ) -> Sm120ForwardConfig:
        if has_bias:
            if max(head_dim, head_dim_v) > 128:
                raise ValueError(
                    "SM120 relative bias currently supports head_dim and "
                    "head_dim_v up to 128"
                )
            if tile_mn is not None and tile_mn != (64, 128):
                raise ValueError("SM120 relative bias requires tile_mn=(64, 128)")
            tile_m, tile_n = 64, 128
        elif tile_mn is None:
            tile_m, tile_n = FlashAttentionForwardSm120.get_fwd_tile_size(
                head_dim,
                head_dim_v,
                total_q_rows=total_q_rows,
                num_sms=num_sms,
                num_batch=num_batch,
                seqlen_q=seqlen_q,
                seqlen_k=seqlen_k,
                num_head_kv=num_head_kv,
                qhead_per_kvhead=qhead_per_kvhead,
                is_causal=is_causal,
                is_local=is_local,
                window_size_left=window_size_left,
                window_size_right=window_size_right,
                pack_gqa=pack_gqa,
                paged_kv=paged_kv,

View on GitHub (pinned to 0132848349)

Solutions

Pass tile_mn=None when has_bias=True and let select_config choose (64, 128)
Or explicitly pass tile_mn=(64, 128)
Exclude bias-enabled runs from autotune configs with other tiles

Example fix

// before
cfg = select_config(..., has_bias=True, tile_mn=(128, 64))
// after
cfg = select_config(..., has_bias=True, tile_mn=None)

Defensive patterns

Strategy: validation

Validate before calling

if has_bias:
    tile_mn = None  # let select_config pick (64, 128)

Prevention

Scope autotuned tile caches by (has_bias, dims, arch) so bias runs never reuse non-bias tiles

When it happens

Trigger: Calling _flash_attn_fwd with has_bias=True and tile_mn explicitly set to anything other than (64, 128), e.g. (128, 64) tuned for a non-bias run.

Common situations: Reusing a hand-tuned tile config from a bias-free benchmark; autotuning sweeps that persist the best tile across configurations with and without bias.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7bf9f001ed5a7a96. Report an issue: GitHub.