sgl-project/sglang · error · ValueError

Invalid arch format: {arch_str}

Error message

Invalid arch format: {arch_str}

What it means

_parse_arch_str parses GPU architecture strings like 'sm_90a', 'SM100', or bare '90' into an integer. Anything not matching ^(?:sm_?)?(\d)(\d)([af]?)$ — bad separators, non-digits, more than one suffix letter — raises this ValueError.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/interface.py:81

    try_cached_paged_decode,
    try_cached_varlen,
)
from sglang.kernels.ops.attention.flash_attn.cute.shearing_bias import ShearingBias

# SM100 head_dim=256 2CTA kernel imports
from sglang.kernels.ops.attention.flash_attn.cute.sm100_hd256_2cta_fmha_forward import (
    BlackwellFusedMultiHeadAttentionForward,
)
from sglang.kernels.ops.attention.flash_attn.cute.utils import AuxData


def _parse_arch_str(arch_str):
    """Parse arch string (e.g. 'sm_80', 'sm_90a', '80', '100') to int (e.g. 80, 90, 100)."""
    import re

    match = re.match(r"^(?:sm_?|SM_?)?(\d+)(\d)([af]?)$", arch_str)
    if not match:
        raise ValueError(f"Invalid arch format: {arch_str}")
    major, minor, _ = match.groups()
    return int(major) * 10 + int(minor)


@lru_cache(maxsize=None)
def _get_device_arch():
    """Cached device arch check.

    Override with FLASH_ATTENTION_ARCH (e.g. 'sm_80' or '80') to select the
    kernel path independently of the compilation target (CUTE_DSL_ARCH).

    For CPU-only compilation (no GPU), set both:
      FLASH_ATTENTION_ARCH=sm_80  (kernel selection)
      CUTE_DSL_ARCH=sm_80         (compilation target)
    """
    arch_override = os.environ.get("FLASH_ATTENTION_ARCH", None)
    if arch_override is not None:
        return _parse_arch_str(arch_override)

View on GitHub (pinned to 0132848349)

Solutions

1. Use the canonical form: 'sm_100', 'sm_100a', 'SM90', or plain '90'/'100'
2. Map vendor names to compute capability (h100->90, b200->100) before passing
3. Check for stray characters/typos in the config value

Example fix

// before
_parse_arch_str('b200')
// after
_parse_arch_str('sm_100')

import re
assert re.match(r'^(?:sm_?)?(\d)(\d)([af]?)$', arch_str), arch_str

def is_valid_arch_str(s: str) -> bool:
    return re.match(r'^(?:sm_?)?(\d)(\d)([af]?)$', s) is not None

try:
    arch = _parse_arch_str(cfg)
except ValueError:
    arch = {'h100': 90, 'a100': 80, 'b200': 100}.get(cfg.lower())  # name map fallback

When it happens

Trigger: Calling _get_device_arch/_parse_arch_str (or an API that derives arch from a string) with malformed input like 'sm-90', '90aa', 'gpu100', or 'b200'.

Common situations: Passing a marketing GPU name (e.g. 'b200', 'h100') instead of the compute-capability form, or a typo in an env var / config that carries the arch string.

Related errors

• Unsupported compute capability: {arch}. Supported: 8.x, 9.x,
• num_token_non_padded and x must be on the same device
• topk_ids must be a CUDA tensor
• backend must be a non-empty string
• Invalid simulate_acc_method: {simulate_acc_method}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/33d9f8d3d39bf3c3. Report an issue: GitHub.