sgl-project/sglang · error · ValueError

(head_dim, head_dim_v)=({head_dim}, {head_dim_v}) exceeds SM

Error message

(head_dim, head_dim_v)=({head_dim}, {head_dim_v}) exceeds SM120 shared-memory capacity ({smem_capacity} bytes)

What it means

get_fwd_tile_size searches tile shapes for the SM120 FlashAttention forward kernel such that the pipeline's shared-memory usage fits the GPU's smem capacity. For very large head_dim/head_dim_v combinations no tile fits, so it raises this ValueError.

Source

Thrown at python/sglang/kernels/ops/attention/fa4_sm120/flash_fwd.py:686

            # packed GQA local attention reaches its steady state.  Keep MHA
            # and larger SM arrays on the cross-SKU HD256 LPT calibration.
            preferred = (64, 48, 1)
            candidates = (preferred,) + tuple(
                candidate for candidate in candidates if candidate != preferred
            )
        for tile_m, tile_n, num_stages in candidates:
            if (
                FlashAttentionForwardSm120._smem_usage_in_bytes(
                    head_dim,
                    head_dim_v,
                    tile_m,
                    tile_n,
                    num_stages,
                )
                <= smem_capacity
            ):
                return tile_m, tile_n
        raise ValueError(
            f"(head_dim, head_dim_v)=({head_dim}, {head_dim_v}) exceeds "
            f"SM120 shared-memory capacity ({smem_capacity} bytes)"
        )

    @staticmethod
    def get_fwd_num_stages(
        head_dim: int, head_dim_v: int, tile_m: int, tile_n: int
    ) -> int:
        """Return the public pipeline specialization depth."""
        return 1

    @staticmethod
    def get_fwd_num_threads(
        head_dim: int,
        head_dim_v: int,
        tile_m: int,
        tile_n: int,
        paged_kv: bool = False,

View on GitHub (pinned to 0132848349)

Solutions

1. Reduce head_dim or head_dim_v (e.g. cap at 128/256) for the SM120 path
2. Route such large-head-dim models to a different attention backend that supports them
3. Query utils to check smem capacity and pre-validate the config instead of crashing at tile search

from sglang.kernels.ops.attention.fa4_sm120 import utils_basic
cap = utils_basic.get_smem_capacity_in_bytes("sm_120")
# pre-check: rough smem need grows ~ (head_dim + head_dim_v); reject > 128+128 configs early

When it happens

Trigger: Requesting a forward tile size with head_dim + head_dim_v so large (e.g. 256+256) that even the smallest tile's smem footprint exceeds the SM120 (Blackwell) shared-memory capacity.

Common situations: Emerging model architectures with head_dim > 128/256; overriding head_dim_v in a custom model; running a config tuned for a datacenter GPU (H100, larger smem) on a consumer Blackwell card with less shared memory.

Related errors

• SM120 relative bias currently supports head_dim and head_dim
• The requested FlashAttention forward configuration exceeds S
• The requested SM120 sheared-bias specialization exceeds shar
• SM120 relative bias requires tile_mn=(64, 128)
• FlashAttention-4 CUTE is not available. Install flash-attn-4

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/869ae8d7316b178f. Report an issue: GitHub.