sgl-project/sglang · error · ValueError

Unsupported compute capability: {arch}. Supported: 8.x, 9.x,

Error message

Unsupported compute capability: {arch}. Supported: 8.x, 9.x, 10.x, 11.x, and architectures registered through the forward-host bridge

What it means

The FA4 CuTe dispatcher only implements kernels for compute capabilities 8.x, 9.x, 10.x, 11.x (plus architectures registered via the forward-host bridge). Any other arch value falls through to this ValueError.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/interface.py:1554

                head_dim=head_dim,
                head_dim_v=head_dim_v,
                qhead_per_kvhead=qhead_per_kvhead,
                is_causal=causal,
                is_local=local,
                pack_gqa=pack_gqa,
                config=arch_forward_config,
                paged_kv=page_table is not None,
                score_mod=score_mod,
                mask_mod=mask_mod,
                has_aux_tensors=aux_tensors is not None,
                is_split_kv=is_split_kv,
                has_bias=bias is not None,
                bias_block_size=tile_bias,
                rel_extent_padded=rel_extent_padded,
                plan=arch_forward_plan,
            )
        else:
            raise ValueError(
                f"Unsupported compute capability: {arch}. Supported: 8.x, 9.x, "
                "10.x, 11.x, and architectures registered through the "
                "forward-host bridge"
            )
        # TODO: check @can_implement
        if qv is not None:
            _flash_attn_fwd.compile_cache[compile_key] = cute.compile(
                fa_fwd,
                q_tensor,
                qv_tensor,
                k_tensor,
                v_tensor,
                o_tensor,
                lse_tensor,
                softmax_scale,
                p_tensor,
                row_max_tensor,
                cu_seqlens_q_tensor,

View on GitHub (pinned to 0132848349)

Solutions

1. Run on a supported GPU: Ampere (8.x), Hopper (9.x), Blackwell (10.x/11.x)
2. If on newer hardware, upgrade sglang/FA4 so the arch is registered or the forward-host bridge covers it
3. Verify torch.cuda.get_device_capability() reports an expected value; check for arch overrides

Example fix

# before: running on T4 (SM75)
out = fa(q, k, v)
# after: run on A100/H100/B200, or
host = get_forward_host(arch); register_forward_host(arch, host)  # custom backend

major, minor = torch.cuda.get_device_capability()
assert (8,9,10,11).__contains__(major) or custom_host_registered(major*10+minor), f'unsupported arch {major}.{minor}'

def arch_supported(arch: int) -> bool: return arch // 10 in (8, 9, 10, 11)

try:
    out = fa(q, k, v)
except ValueError as e:
    if 'Unsupported compute capability' in str(e):
        out = fallback_attention(q, k, v)  # e.g. torch SDPA
    else:
        raise

When it happens

Trigger: Running _flash_attn_fwd on a GPU with an unrecognized compute capability — e.g. an old pre-Ampere card (7.5), an unregistered new arch, or a bogus arch integer derived from a malformed arch string/env override.

Common situations: Running on Turing (T4, 20xx) or older; using a torch/CUDA build that reports an unexpected arch; forcing an arch via env var/forward-host bridge that wasn't registered.

Related errors

• SplitKV partial output (mO) must be Float32
• All tensors must have the same data type
• Only Float16 or BFloat16 is supported
• LSE tensor must be Float32
• cu_seqlens_q tensor must be Int32

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0322f3e406f8ad48. Report an issue: GitHub.