sgl-project/sglang · error · ValueError

TRTLLM MHA backend for prefill requires Hopper (SM90), Black

Error message

TRTLLM MHA backend for prefill requires Hopper (SM90), Blackwell (SM100), or SM120 GPUs. Please use a different prefill backend.

What it means

The trtllm_mha prefill backend's kernels are only compiled for Hopper (SM90), Blackwell (SM100), and SM120 GPUs. During server-arg resolution SGLang probes the current GPU's compute capability and refuses to select trtllm_mha for prefill on older architectures.

Source

Thrown at python/sglang/srt/server_args.py:6557

        # resolution pipeline (arg_groups/overrides.py:
        # _mla_kv_cache_dtype_checks), invoked here at their legacy slot.
        from sglang.srt.arg_groups.overrides import _mla_kv_cache_dtype_checks

        run_post_process_pass(self, _mla_kv_cache_dtype_checks)

        # The CuteDSL MLA validation + prefill fill moved to the resolution
        # pipeline (arg_groups/overrides.py: _cutedsl_prefill_backend_fill),
        # invoked here at its legacy slot.
        from sglang.srt.arg_groups.overrides import _cutedsl_prefill_backend_fill

        run_post_process_pass(self, _cutedsl_prefill_backend_fill)

        prefill_backend, decode_backend = self._resolved_attention_backends()
        if "trtllm_mha" in (prefill_backend, decode_backend):
            if prefill_backend == "trtllm_mha" and not (
                is_sm90_supported() or is_sm100_supported() or is_sm120_supported()
            ):
                raise ValueError(
                    "TRTLLM MHA backend for prefill requires Hopper (SM90), Blackwell (SM100), or SM120 GPUs. "
                    "Please use a different prefill backend."
                )
            if (
                prefill_backend == "trtllm_mha"
                and is_sm120_supported()
                and (
                    cfg.kv_cache_dtype == "fp8_e4m3"
                    or (
                        envs.SGLANG_SKIP_SOFTMAX_PREFILL_THRESHOLD_SCALE_FACTOR.get()
                        or 0.0
                    )
                    > 0
                )
            ):
                raise ValueError(
                    "TRTLLM FMHAv2 prefill on SM120 does not support "
                    "fp8_e4m3 KV cache or skip-softmax."

View on GitHub (pinned to 0132848349)

Solutions

Switch prefill to a portable backend: --prefill-attention-backend fa3 (Hopper) or triton/flashinfer
Verify the GPU first: nvidia-smi or python -c "import torch;print(torch.cuda.get_device_capability())"
Run on SM90/SM100/SM120 hardware if trtllm_mha prefill is required

Example fix

# before
python -m sglang.launch_server --model M --prefill-attention-backend trtllm_mha
# after
python -m sglang.launch_server --model M --prefill-attention-backend triton --decode-attention-backend trtllm_mha

Defensive patterns

Strategy: validation

Validate before calling

import torch
cap = torch.cuda.get_device_capability(0)
sm = cap[0] * 10 + cap[1]
if args.prefill_attention_backend == "trtllm_mha" and sm not in (90, 100, 120):
    args.prefill_attention_backend = "triton"  # or fa3 on SM90

Try / catch

try:
    ServerArgs(**kwargs)
except ValueError as e:
    if "TRTLLM MHA backend for prefill" in str(e):
        kwargs["prefill_attention_backend"] = "triton"
        ServerArgs(**kwargs)
    else:
        raise

Prevention

Query torch.cuda.get_device_capability() in launch scripts and select backends programmatically
Gate trtllm_mha behind an SM90/SM100/120 check in deployment templates
Run a dry arg-resolution pass before committing GPU jobs

When it happens

Trigger: Passing --prefill-attention-backend trtllm_mha (or --attention-backend trtllm_mha) on a GPU where is_sm90_supported(), is_sm100_supported(), and is_sm120_supported() all return false (Ampere/Ada/older).

Common situations: Reusing an H100/B200-tuned launch script on an A100 or L40S machine; CI runners with older GPUs; a default backend profile silently resolving prefill to trtllm_mha on unsupported hardware.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b3cfbca4a94b25c0. Report an issue: GitHub.