{"record":{"id":"b813d63c19a174ee","repo":"sgl-project/sglang","slug":"flashinfer-allreduce-fusion-mnnvl-backend-requires","errorCode":null,"errorMessage":"FlashInfer allreduce fusion mnnvl backend requires a Blackwell system, or SM90 single-node.","messagePattern":"FlashInfer allreduce fusion mnnvl backend requires a Blackwell system, or SM90 single-node\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/flashinfer_comm_fusion.py","lineNumber":71,"sourceCode":"    if backend == \"auto\":\n        if is_multi_node:\n            if is_sm100_supported():\n                return \"mnnvl\"\n            raise ValueError(\n                \"FlashInfer allreduce fusion does not support multi-node on \"\n                \"non-Blackwell systems.\"\n            )\n        if is_sm100_supported():\n            return \"mnnvl\"\n        return \"trtllm\"\n\n    if backend == \"trtllm\" and is_multi_node:\n        raise ValueError(\n            \"FlashInfer allreduce fusion trtllm backend supports single-node only.\"\n        )\n\n    if backend == \"mnnvl\" and not _mnnvl_supported(is_multi_node):\n        raise ValueError(\n            \"FlashInfer allreduce fusion mnnvl backend requires a Blackwell \"\n            \"system, or SM90 single-node.\"\n        )\n    return backend\n\n\ndef resolve_flashinfer_allreduce_fusion_backend() -> Optional[str]:\n    \"\"\"The fusion backend for this process, or None when fusion is off.\n\n    Reads the published leaves (`exec.comm`, `parallel`): the backend is a\n    resolution decision, and the node count is launch topology.\n    \"\"\"\n    backend = get_exec().comm.flashinfer_allreduce_fusion_backend\n    if backend is None:\n        return None\n    return _resolve_backend(backend, get_parallel().nnodes > 1)\n\n","sourceCodeStart":53,"sourceCodeEnd":89,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/flashinfer_comm_fusion.py#L53-L89","documentation":"FlashInfer allreduce fusion supports several backends; the 'mnnvl' (multi-node NVLink) backend requires Blackwell GPUs, or a single-node SM90 (Hopper) system. _resolve_backend validates the chosen backend against the topology and GPU architecture before returning it, raising ValueError when mnnvl is requested on unsupported hardware such as pre-Hopper GPUs or a multi-node non-Blackwell cluster.","triggerScenarios":"Passing --enable-flashinfer-allreduce-fusion (or configuring backend='mnnvl' in resolve_flashinfer_allreduce_fusion_backend) on a system where _mnnvl_supported(is_multi_node) is False: e.g. multi-node Hopper (SM90) cluster, or any Ampere/Ada (SM80/SM89) single- or multi-node system.","commonSituations":"Enabling flashinfer allreduce fusion flags copied from a Blackwell (B200/GB200) setup onto an H100 multi-node cluster or A100 machines; defaulting the backend to mnnvl without checking torch.cuda.get_device_capability().","solutions":["Switch to the 'trtllm' backend on single-node systems (it supports single-node only), or disable flashinfer allreduce fusion","Upgrade to Blackwell hardware if multi-node mnnvl allreduce fusion is required","Verify GPU architecture with torch.cuda.get_device_capability() and only request mnnvl when major >= 100 (Blackwell) or (major == 90 and single-node)"],"exampleFix":"# before\nbackend = \"mnnvl\"  # on multi-node H100 cluster\n\n# after\nmajor, _ = torch.cuda.get_device_capability()\nbackend = \"mnnvl\" if major >= 100 else (\"trtllm\" if single_node else None)","handlingStrategy":"validation","validationCode":"import torch\nmajor, _ = torch.cuda.get_device_capability()\nsingle_node = get_world_size() == torch.cuda.device_count()\ndef mnnvl_ok(single_node): return major >= 100 or (major == 90 and single_node)\nbackend = \"mnnvl\" if mnnvl_ok(single_node) else \"trtllm\" if single_node else None","typeGuard":null,"tryCatchPattern":"try:\n    backend = resolve_flashinfer_allreduce_fusion_backend(backend=\"mnnvl\", is_multi_node=multi)\nexcept ValueError as e:\n    if \"mnnvl backend requires\" in str(e):\n        logger.warning(\"mnnvl unsupported here; falling back to trtllm/none\")\n        backend = \"trtllm\" if not multi else None\n    else:\n        raise","preventionTips":["Gate backend selection on torch.cuda.get_device_capability() and node count","Never hardcode mnnvl in shared launch scripts","Test new fusion flags on one node with the target GPU arch before fleet rollout"],"tags":["flashinfer","allreduce-fusion","gpu-architecture","mnnvl","backend-selection"],"backgroundTag":"unsupported-hardware-capability","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}