{"record":{"id":"e1c3b65bb0eeb544","repo":"vllm-project/vllm","slug":"flashinfer-allreduce-is-not-supported-for-multi-no","errorCode":null,"errorMessage":"Flashinfer allreduce is not supported for multi-node allreduce with 'trtllm' backend. Please use 'mnnvl' backend instead.","messagePattern":"Flashinfer allreduce is not supported for multi-node allreduce with 'trtllm' backend\\. Please use 'mnnvl' backend instead\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/distributed/device_communicators/flashinfer_all_reduce.py","lineNumber":163,"sourceCode":"    hidden_dim: int,\n    dtype: torch.dtype,\n    group: ProcessGroup,\n):\n    \"\"\"\n    Return the allreduce workspace for non-quant patterns, initializing if needed.\n\n    Used by AllReduceFusionPass (non-quant patterns) and FlashInferAllReduce\n    for standalone allreduce. Backend is controlled by\n    VLLM_FLASHINFER_ALLREDUCE_BACKEND env var.\n    \"\"\"\n    global _fi_ar_workspace\n    if _fi_ar_workspace is not None:\n        return _fi_ar_workspace\n\n    backend, allow_trtllm_fallback = _resolve_fi_ar_backend()\n\n    if get_node_count() > 1 and backend == \"trtllm\":\n        raise ValueError(\n            \"Flashinfer allreduce is not supported for multi-node allreduce with \"\n            \"'trtllm' backend. Please use 'mnnvl' backend instead.\"\n        )\n\n    def _get_or_create(be: str):\n        # Reuse the quant workspace if it was already created with the same backend\n        if _fi_ar_quant_workspace is not None and _fi_ar_quant_workspace.backend == be:\n            return _fi_ar_quant_workspace\n        return _create_workspace(\n            be, world_size, rank, max_token_num, hidden_dim, dtype, group\n        )\n\n    _fi_ar_workspace = _get_or_create(backend)\n    if _fi_ar_workspace is None and allow_trtllm_fallback and backend != \"trtllm\":\n        logger.warning_once(\n            \"FlashInfer mnnvl allreduce workspace unavailable (likely no NVSwitch \"\n            \"multicast support); falling back to trtllm backend for single node.\"\n        )","sourceCodeStart":145,"sourceCodeEnd":181,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/device_communicators/flashinfer_all_reduce.py#L145-L181","documentation":"get_flashinfer_allreduce_workspace builds the FlashInfer allreduce workspace for the non-quant fusion path. The backend is chosen from VLLM_FLASHINFER_ALLREDUCE_BACKEND (or 'auto'). The 'trtllm' allreduce kernel only works within a single node, so the guard raises ValueError when get_node_count() > 1 and the resolved backend is 'trtllm'; multi-node requires the 'mnnvl' multicast backend.","triggerScenarios":"Running a multi-node vLLM deployment (tensor parallel across nodes, get_node_count() > 1) with VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm explicitly set (or auto-resolved to trtllm) while a graph with allreduce fusion is prepared, which calls get_flashinfer_allreduce_workspace.","commonSituations":"Copying a single-node config (which pinned trtllm) to a multi-node Ray/slurm deployment; enabling --enable-flashinfer-allreduce-fusion on nodes without NVLink multicast so 'auto' falls back to trtllm; upgrading to a topology where the previous backend choice no longer applies.","solutions":["Set VLLM_FLASHINFER_ALLREDUCE_BACKEND=mnnvl for the multi-node deployment","If nodes lack multicast NVLink, disable FlashInfer allreduce fusion and fall back to the default PyNccl allreduce path","Restrict the trtllm backend to single-node topologies only"],"exampleFix":"# before\nexport VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm\n# multi-node launch -> ValueError\n\n# after\nexport VLLM_FLASHINFER_ALLREDUCE_BACKEND=mnnvl\n# or unset + disable fusion to use default allreduce","handlingStrategy":"validation","validationCode":"import os\nfrom vllm.distributed import get_node_count\nbackend = os.environ.get(\"VLLM_FLASHINFER_ALLREDUCE_BACKEND\", \"auto\")\nif get_node_count() > 1:\n    assert backend in (\"mnnvl\", \"auto\"), \"multi-node flashinfer allreduce requires mnnvl backend\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never pin VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm in shared configs","Set backend per topology in launch scripts","Smoke-test one allreduce-fused step before long runs"],"tags":["distributed","flashinfer","allreduce","multi-node","config"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}