sgl-project/sglang · error · ValueError

The requested FlashAttention forward configuration exceeds S

Error message

The requested FlashAttention forward configuration exceeds SM120 kernel constraints or shared-memory capacity

What it means

make_kernel validates the requested FlashAttention SM120 configuration (tile sizes, num_stages, num_threads, causal, paged KV) against kernel constraints and shared-memory capacity via a predicate call. If the combination is infeasible it raises this generic ValueError.

Source

Thrown at python/sglang/kernels/ops/attention/fa4_sm120/runtime.py:961

        is_split_kv: bool,
        has_bias: bool,
        bias_block_size: int,
        rel_extent_padded: int,
        plan: Sm120ForwardPlan,
    ) -> FlashAttentionForwardSm120:
        if not FlashAttentionForwardSm120.can_implement(
            dtype,
            head_dim,
            head_dim_v,
            config.tile_m,
            config.tile_n,
            num_stages=config.num_stages,
            num_threads=config.num_threads,
            is_causal=is_causal,
            Q_in_regs=False,
            paged_kv=paged_kv,
        ):
            raise ValueError(
                "The requested FlashAttention forward configuration exceeds "
                "SM120 kernel constraints or shared-memory capacity"
            )
        if has_bias:
            bias_smem_bytes = (
                bias_block_size * config.tile_n * (dtype.width // 8) * config.num_stages
            )
            total_smem_bytes = (
                FlashAttentionForwardSm120._smem_usage_in_bytes(
                    head_dim,
                    head_dim_v,
                    config.tile_m,
                    config.tile_n,
                    config.num_stages,
                    False,
                )
                + bias_smem_bytes
            )

View on GitHub (pinned to 0132848349)

Solutions

Pass config=None (or the equivalent) to let the runtime auto-select a feasible config
Re-run autotuning on the target GPU so cached configs match its constraints
Reduce num_stages or tile size in the requested config

Defensive patterns

Strategy: fallback

Validate before calling

try:
    Kernel = make_kernel(config=cfg, ...)
except ValueError:
    Kernel = make_kernel(config=None, ...)  # auto-select

Try / catch

try:
    Kernel = make_kernel(config=cfg, ...)
except ValueError as e:
    logger.warning("config infeasible on this GPU, falling back to autotune: %s", e)
    Kernel = make_kernel(config=None, ...)

Prevention

Re-autotune on each GPU SKU; never share config caches across architectures
Prefer config=None unless you have measured a specific config on this device

When it happens

Trigger: Building the kernel with an explicit config whose tile/stage/thread combination fails the internal feasibility check — e.g. large tiles with many stages, or a config copied from a different GPU/architecture.

Common situations: Loading persisted autotune configs tuned on a different SM120 SKU with more shared memory; manual tile overrides; num_stages too high for the chosen tile.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/56c133e096c06549. Report an issue: GitHub.