sgl-project/sglang · error · ValueError

--dcp-comm-backend fi_a2a delegates the exchange to FlashInf

Error message

--dcp-comm-backend fi_a2a delegates the exchange to FlashInfer's MNNVL All-to-All kernel, which requires an NVIDIA CUDA platform with SM90+ and MNNVL fabric memory (e.g. GB200 NVL72). The authoritative fabric probe runs at model-runner init; use 'a2a' or 'ag_rs' on clusters without MNNVL.

What it means

ServerArgs validation error raised when --dcp-comm-backend is set to fi_a2a on a non-CUDA platform. fi_a2a delegates the decode context-parallel token exchange to FlashInfer's MNNVL All-to-All kernel, which only exists on NVIDIA SM90+ hardware with MNNVL fabric memory (e.g. GB200 NVL72). SGLang rejects it up front because the kernel cannot run elsewhere.

Source

Thrown at python/sglang/srt/server_args.py:4269

        handle_pd_disaggregation(self)

    def _handle_dcp_validation(self):
        cfg = resolving_view(self)
        if cfg.dcp_size < 1:
            raise ValueError(
                "Decode context parallel size (--dcp-size / "
                "--decode-context-parallel-size) must be >= 1, but got "
                f"dcp_size={cfg.dcp_size}."
            )
        if cfg.dcp_comm_backend in ("a2a", "fi_a2a") and cfg.dcp_size <= 1:
            raise ValueError(
                f"--dcp-comm-backend {cfg.dcp_comm_backend} only affects the "
                "decode context-parallel attention reduction and therefore "
                "requires --dcp-size / --decode-context-parallel-size > 1, but "
                f"got dcp_size={cfg.dcp_size}."
            )
        if cfg.dcp_comm_backend == "fi_a2a" and not is_cuda():
            raise ValueError(
                "--dcp-comm-backend fi_a2a delegates the exchange to FlashInfer's "
                "MNNVL All-to-All kernel, which requires an NVIDIA CUDA platform "
                "with SM90+ and MNNVL fabric memory (e.g. GB200 NVL72). The "
                "authoritative fabric probe runs at model-runner init; use 'a2a' "
                "or 'ag_rs' on clusters without MNNVL."
            )
        if cfg.dcp_replicate_q_proj:
            if cfg.dcp_size <= 1:
                raise ValueError("--dcp-replicate-q-proj requires --dcp-size > 1.")
            if cfg.dcp_comm_backend not in ("a2a", "fi_a2a"):
                raise ValueError(
                    "--dcp-replicate-q-proj only applies to the a2a/fi_a2a DCP "
                    "communication backend (it removes the head-dim Q all-gather); "
                    f"got --dcp-comm-backend={cfg.dcp_comm_backend}."
                )

    def _handle_load_balance_method(self):
        cfg = resolving_view(self)

View on GitHub (pinned to 0132848349)

Solutions

1. Switch to a portable backend: --dcp-comm-backend a2a or --dcp-comm-backend ag_rs
2. Only keep fi_a2a on SM90+ machines with MNNVL fabric memory (GB200 NVL72); verify with nvidia-smi architecture and fabric visibility
3. Gate the flag in your launch script on the detected platform (torch.cuda.is_available() and compute capability >= 9.0)

Example fix

# before
python -m sglang.launch_server --dcp-size 4 --dcp-comm-backend fi_a2a
# after
python -m sglang.launch_server --dcp-size 4 --dcp-comm-backend a2a

import torch

def dcp_backend_ok() -> str:
    cuda = torch.cuda.is_available()
    cap = torch.cuda.get_device_capability(0)[0] if cuda else 0
    return "fi_a2a" if cuda and cap >= 9 else "a2a"

When it happens

Trigger: Launching the server with --dcp-comm-backend fi_a2a (and --dcp-size > 1) on a machine where is_cuda() is false, e.g. AMD ROCm, CPU-only, or any non-NVIDIA accelerator platform.

Common situations: Porting a DCP configuration from a GB200/NVL72 cluster to an A100/H100 or AMD cluster; test configs copied from MNNVL docs; running unit tests on non-CUDA CI runners with the flag set.

Related errors

• Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requ
• Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requ
• --dcp-replicate-q-proj requires --dcp-size > 1.
• --dcp-replicate-q-proj only applies to the a2a/fi_a2a DCP co
• {name} is deprecated; use {replacement} instead.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/761451d799a5d8de. Report an issue: GitHub.