sgl-project/sglang · error · ImportError
NPU detected, but torchair package is not installed. Please…
Error message
NPU detected, but torchair package is not installed. Please install torchair for torch.compile support on NPU.
What it means
Raised when SGLang detects an available NPU (torch.npu.is_available()) and tries to build a torch.compile backend, but the torchair package (or its submodules) cannot be imported. torchair is Huawei's Ascend NPU compiler integration needed for torch.compile on NPU hardware. Without it, the npugraph/npugraph_ex compiler modes cannot function.
Solutions
- pip install a torchair version matching your CANN and torch builds (see Ascend pytorch and torchair release matrix)
- Verify import works: python -c "import torchair; import torchair.ge_concrete_graph.ge_converter.experimental.patch_for_hcom_allreduce; from torchair.configs.compiler_config import CompilerConfig"
- If you don't need torch.compile on NPU, disable it (drop --enable-torch-compile / npugraph modes)
- Upgrade/downgrade torch to the version the installed torchair was compiled against
Example fix
# before server_args.enable_torch_compile = True # on NPU without torchair # after pip install torchair==<version matching your CANN/torch> # or disable compile on NPU server_args.enable_torch_compile = False
Defensive patterns
Strategy: validation
Validate before calling
def has_torchair() -> bool:
try:
import torchair
import torchair.ge_concrete_graph.ge_converter.experimental.patch_for_hcom_allreduce # noqa
from torchair.configs.compiler_config import CompilerConfig
return True
except ImportError:
return False
if torch.npu.is_available() and args.enable_torch_compile and not has_torchair():
args.enable_torch_compile = False # or abort with a clear message Prevention
- Pin a matching torch/torchair/CANN version set in your environment manifest
- Add a startup smoke test that imports torchair on NPU nodes before serving
- Gate --enable-torch-compile on NPU behind a capability check
When it happens
Trigger: Calling get_compiler_backend('npugraph' or 'npugraph_ex') on a machine where torch.npu.is_available() is True but torchair, its hcom_allreduce patch module, or torchair.configs.compiler_config.CompilerConfig is missing/mismatched. Reached via _maybe_enable_torch_compile, build_torch_compile_kwargs, patch_model_npu, or model classes like Llama4MoE that hardcode the NPU backend.
Common situations: Running SGLang on Ascend NPU with a torch/torchair version mismatch; torchair installed but too old to have ge_converter.experimental.patch_for_hcom_allreduce; enabling --enable-torch-compile on NPU without installing the CANN-matched torchair wheel.
Understand the failure class
Background: "X is not installed. Please install it with pip install Y": missing optional dependency errors — ImportError/ValueError raised when a library's optional extra was never installed — this error's family across 22 libraries.
Related errors
- The required 'attentions' package is not installed. Install…
- Ascend A2/A3 NPU does not support nvfp4…
- Ascend PD transfer does not support HiSparse destination…
- Ascend Transfer Engine initialization failed.
- combine() called before dispatch()
AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28).
Data as JSON: /api/errors/0e6b4ec812762bea.
Report an issue: GitHub.
Appendix: source
Thrown at python/sglang/srt/utils/common.py:1003
return major, minor
def get_compiler_backend(mode=None) -> str:
# OOT platforms provide their own compile backend.
if current_platform.is_out_of_tree():
return current_platform.get_compile_backend(mode)
if hasattr(torch, "hpu") and torch.hpu.is_available():
return "hpu_backend"
if hasattr(torch, "npu") and torch.npu.is_available():
try:
import torchair
import torchair.ge_concrete_graph.ge_converter.experimental.patch_for_hcom_allreduce # noqa: F401
from torchair.configs.compiler_config import CompilerConfig
except ImportError:
raise ImportError(
"NPU detected, but torchair package is not installed. "
"Please install torchair for torch.compile support on NPU."
)
compiler_config = CompilerConfig()
compiler_config.mode = "max-autotune"
if mode == "npugraph_ex":
compiler_config.mode = "reduce-overhead"
compiler_config.debug.run_eagerly = True
npu_backend = torchair.get_npu_backend(compiler_config=compiler_config)
return npu_backend
return "inductor"
def set_cuda_arch():
if is_flashinfer_available():
capability = torch.cuda.get_device_capability()
arch = f"{capability[0]}.{capability[1]}"View on GitHub (pinned to 0132848349)