sgl-project/sglang · error · ImportError

NPU detected, but torchair package is not installed. Please

Error message

NPU detected, but torchair package is not installed. Please install torchair for torch.compile support on NPU.

What it means

Raised when SGLang detects an available NPU (torch.npu.is_available()) and tries to build a torch.compile backend, but the torchair package (or its submodules) cannot be imported. torchair is Huawei's Ascend NPU compiler integration needed for torch.compile on NPU hardware. Without it, the npugraph/npugraph_ex compiler modes cannot function.

Source

Thrown at python/sglang/srt/utils/common.py:1003

    return major, minor


def get_compiler_backend(mode=None) -> str:
    # OOT platforms provide their own compile backend.
    if current_platform.is_out_of_tree():
        return current_platform.get_compile_backend(mode)

    if hasattr(torch, "hpu") and torch.hpu.is_available():
        return "hpu_backend"

    if hasattr(torch, "npu") and torch.npu.is_available():
        try:
            import torchair
            import torchair.ge_concrete_graph.ge_converter.experimental.patch_for_hcom_allreduce  # noqa: F401
            from torchair.configs.compiler_config import CompilerConfig
        except ImportError:
            raise ImportError(
                "NPU detected, but torchair package is not installed. "
                "Please install torchair for torch.compile support on NPU."
            )
        compiler_config = CompilerConfig()
        compiler_config.mode = "max-autotune"
        if mode == "npugraph_ex":
            compiler_config.mode = "reduce-overhead"
            compiler_config.debug.run_eagerly = True
        npu_backend = torchair.get_npu_backend(compiler_config=compiler_config)
        return npu_backend

    return "inductor"


def set_cuda_arch():
    if is_flashinfer_available():
        capability = torch.cuda.get_device_capability()
        arch = f"{capability[0]}.{capability[1]}"

View on GitHub (pinned to 0132848349)

Solutions

1. pip install a torchair version matching your CANN and torch builds (see Ascend pytorch and torchair release matrix)
2. Verify import works: python -c "import torchair; import torchair.ge_concrete_graph.ge_converter.experimental.patch_for_hcom_allreduce; from torchair.configs.compiler_config import CompilerConfig"
3. If you don't need torch.compile on NPU, disable it (drop --enable-torch-compile / npugraph modes)
4. Upgrade/downgrade torch to the version the installed torchair was compiled against

Example fix

# before
server_args.enable_torch_compile = True  # on NPU without torchair
# after
pip install torchair==<version matching your CANN/torch>
# or disable compile on NPU
server_args.enable_torch_compile = False

def has_torchair() -> bool:
    try:
        import torchair
        import torchair.ge_concrete_graph.ge_converter.experimental.patch_for_hcom_allreduce  # noqa
        from torchair.configs.compiler_config import CompilerConfig
        return True
    except ImportError:
        return False

if torch.npu.is_available() and args.enable_torch_compile and not has_torchair():
    args.enable_torch_compile = False  # or abort with a clear message

When it happens

Trigger: Calling get_compiler_backend('npugraph' or 'npugraph_ex') on a machine where torch.npu.is_available() is True but torchair, its hcom_allreduce patch module, or torchair.configs.compiler_config.CompilerConfig is missing/mismatched. Reached via _maybe_enable_torch_compile, build_torch_compile_kwargs, patch_model_npu, or model classes like Llama4MoE that hardcode the NPU backend.

Common situations: Running SGLang on Ascend NPU with a torch/torchair version mismatch; torchair installed but too old to have ge_converter.experimental.patch_for_hcom_allreduce; enabling --enable-torch-compile on NPU without installing the CANN-matched torchair wheel.

Related errors

• The required 'attentions' package is not installed. Install
• {name} is required for NPU packed attention
• {name} must be a 1D int32 or int64 tensor
• {name} and its host copy must have the same length
• {name} must start with 0 and contain at least one sequence

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0e6b4ec812762bea. Report an issue: GitHub.