sgl-project/sglang · error · ValueError

The requested SM120 sheared-bias specialization exceeds shar

Error message

The requested SM120 sheared-bias specialization exceeds shared-memory capacity

What it means

When the sheared-bias specialization is active, make_kernel adds bias_smem_bytes (bias block size x tile_n x dtype width x num_stages) on top of the base kernel's shared-memory usage. If the total exceeds the SM120 shared-memory capacity, it raises this ValueError.

Source

Thrown at python/sglang/kernels/ops/attention/fa4_sm120/runtime.py:981

                "SM120 kernel constraints or shared-memory capacity"
            )
        if has_bias:
            bias_smem_bytes = (
                bias_block_size * config.tile_n * (dtype.width // 8) * config.num_stages
            )
            total_smem_bytes = (
                FlashAttentionForwardSm120._smem_usage_in_bytes(
                    head_dim,
                    head_dim_v,
                    config.tile_m,
                    config.tile_n,
                    config.num_stages,
                    False,
                )
                + bias_smem_bytes
            )
            if total_smem_bytes > utils_basic.get_smem_capacity_in_bytes("sm_120"):
                raise ValueError(
                    "The requested SM120 sheared-bias specialization exceeds "
                    "shared-memory capacity"
                )
        Kernel = FlashAttentionForwardSm120
        if plan.transpose_qk_pv:
            from sglang.kernels.ops.attention.fa4_sm120.flash_fwd_decode import (
                FlashAttentionForwardSm120DecodeTranspose,
            )

            Kernel = FlashAttentionForwardSm120DecodeTranspose
        return Kernel(
            dtype,
            head_dim,
            head_dim_v,
            qhead_per_kvhead,
            is_causal=is_causal,
            is_local=is_local,
            pack_gqa=pack_gqa,

View on GitHub (pinned to 0132848349)

Solutions

1. Reduce bias_block_size or num_stages in the config
2. Use a narrower bias dtype if the kernel supports it
3. Fall back to the non-sheared bias path or a different backend for very large bias footprints

from sglang.kernels.ops.attention.fa4_sm120 import utils_basic
cap = utils_basic.get_smem_capacity_in_bytes("sm_120")
est = bias_block_size * tile_n * (dtype_bits // 8) * num_stages
if est > cap * 0.8:  # leave room for base kernel smem
    num_stages = 2  # or shrink bias_block_size

When it happens

Trigger: Using the sheared-bias path with large bias_block_size, wide bias dtype, many num_stages, or tile_n=128 such that the bias buffer alone blows the smem budget.

Common situations: Large bias block sizes for long-context relative bias; configs tuned without bias re-tuned with bias enabled; dtype promotions (fp32 bias) doubling smem needs.

Related errors

• (head_dim, head_dim_v)=({head_dim}, {head_dim_v}) exceeds SM
• SM120 relative bias currently supports head_dim and head_dim
• SM120 relative bias requires tile_mn=(64, 128)
• The requested FlashAttention forward configuration exceeds S
• FlashAttention-4 CUTE is not available. Install flash-attn-4

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5baac586b78d3bcb. Report an issue: GitHub.