sgl-project/sglang · error · ValueError

FlashInfer allreduce fusion mnnvl backend requires a Blackwe

Error message

FlashInfer allreduce fusion mnnvl backend requires a Blackwell system, or SM90 single-node.

What it means

FlashInfer allreduce fusion supports several backends; the 'mnnvl' (multi-node NVLink) backend requires Blackwell GPUs, or a single-node SM90 (Hopper) system. _resolve_backend validates the chosen backend against the topology and GPU architecture before returning it, raising ValueError when mnnvl is requested on unsupported hardware such as pre-Hopper GPUs or a multi-node non-Blackwell cluster.

Source

Thrown at python/sglang/srt/layers/flashinfer_comm_fusion.py:71

    if backend == "auto":
        if is_multi_node:
            if is_sm100_supported():
                return "mnnvl"
            raise ValueError(
                "FlashInfer allreduce fusion does not support multi-node on "
                "non-Blackwell systems."
            )
        if is_sm100_supported():
            return "mnnvl"
        return "trtllm"

    if backend == "trtllm" and is_multi_node:
        raise ValueError(
            "FlashInfer allreduce fusion trtllm backend supports single-node only."
        )

    if backend == "mnnvl" and not _mnnvl_supported(is_multi_node):
        raise ValueError(
            "FlashInfer allreduce fusion mnnvl backend requires a Blackwell "
            "system, or SM90 single-node."
        )
    return backend


def resolve_flashinfer_allreduce_fusion_backend() -> Optional[str]:
    """The fusion backend for this process, or None when fusion is off.

    Reads the published leaves (`exec.comm`, `parallel`): the backend is a
    resolution decision, and the node count is launch topology.
    """
    backend = get_exec().comm.flashinfer_allreduce_fusion_backend
    if backend is None:
        return None
    return _resolve_backend(backend, get_parallel().nnodes > 1)

View on GitHub (pinned to 0132848349)

Solutions

Switch to the 'trtllm' backend on single-node systems (it supports single-node only), or disable flashinfer allreduce fusion
Upgrade to Blackwell hardware if multi-node mnnvl allreduce fusion is required
Verify GPU architecture with torch.cuda.get_device_capability() and only request mnnvl when major >= 100 (Blackwell) or (major == 90 and single-node)

Example fix

# before
backend = "mnnvl"  # on multi-node H100 cluster

# after
major, _ = torch.cuda.get_device_capability()
backend = "mnnvl" if major >= 100 else ("trtllm" if single_node else None)

Defensive patterns

Strategy: validation

Validate before calling

import torch
major, _ = torch.cuda.get_device_capability()
single_node = get_world_size() == torch.cuda.device_count()
def mnnvl_ok(single_node): return major >= 100 or (major == 90 and single_node)
backend = "mnnvl" if mnnvl_ok(single_node) else "trtllm" if single_node else None

Try / catch

try:
    backend = resolve_flashinfer_allreduce_fusion_backend(backend="mnnvl", is_multi_node=multi)
except ValueError as e:
    if "mnnvl backend requires" in str(e):
        logger.warning("mnnvl unsupported here; falling back to trtllm/none")
        backend = "trtllm" if not multi else None
    else:
        raise

Prevention

Gate backend selection on torch.cuda.get_device_capability() and node count
Never hardcode mnnvl in shared launch scripts
Test new fusion flags on one node with the target GPU arch before fleet rollout

When it happens

Trigger: Passing --enable-flashinfer-allreduce-fusion (or configuring backend='mnnvl' in resolve_flashinfer_allreduce_fusion_backend) on a system where _mnnvl_supported(is_multi_node) is False: e.g. multi-node Hopper (SM90) cluster, or any Ampere/Ada (SM80/SM89) single- or multi-node system.

Common situations: Enabling flashinfer allreduce fusion flags copied from a Blackwell (B200/GB200) setup onto an H100 multi-node cluster or A100 machines; defaulting the backend to mnnvl without checking torch.cuda.get_device_capability().

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b813d63c19a174ee. Report an issue: GitHub.