sgl-project/sglang · error · RuntimeError

torch_npu detected, but NPU device is not available or visib

Error message

torch_npu detected, but NPU device is not available or visible.

What it means

is_npu() detects Ascend NPU support: if the torch_npu module is loaded (torch has an 'npu' attribute) but torch.npu.is_available() is False — no visible NPU device — it raises RuntimeError instead of silently returning False, because a half-installed NPU environment is a misconfiguration.

Source

Thrown at python/sglang/srt/utils/common.py:185

def register_xpu_device_properties_for_dynamo() -> None:
    if not is_xpu():
        return

    import torch._dynamo.utils as dynamo_utils

    xpu_props_type = getattr(torch.xpu, "_XpuDeviceProperties", None)
    if xpu_props_type is not None:
        dynamo_utils.common_constant_types.add(xpu_props_type)


@lru_cache(maxsize=1)
def is_npu() -> bool:
    if not hasattr(torch, "npu"):
        return False

    if not torch.npu.is_available():
        raise RuntimeError(
            "torch_npu detected, but NPU device is not available or visible."
        )

    return True


@lru_cache(maxsize=1)
def is_host_cpu_x86() -> bool:
    machine = platform.machine().lower()
    return (
        machine in ("x86_64", "amd64", "i386", "i686")
        and hasattr(torch, "cpu")
        and torch.cpu.is_available()
    )


def is_host_cpu_arm64() -> bool:
    machine = platform.machine().lower()

View on GitHub (pinned to 0132848349)

Solutions

1. Set ASCEND_RT_VISIBLE_DEVICES to a valid NPU id and verify with npu-smi info
2. If you don't intend NPU, uninstall torch_npu or prevent its import (it's monkeypatching torch) to fall back to CUDA/CPU
3. Fix container/device permissions so the NPU is visible

Example fix

# before
# torch_npu installed, no NPU hardware -> RuntimeError on is_npu()
# after
pip uninstall torch_npu  # or: export ASCEND_RT_VISIBLE_DEVICES=0 with NPU present

import torch
has_npu_mod = hasattr(torch, 'npu')
use_npu = has_npu_mod and torch.npu.is_available() and torch.npu.device_count() > 0

def npu_ready() -> bool:
    return hasattr(torch, 'npu') and torch.npu.is_available()

try:
    from sglang.srt.utils.common import is_npu
    npu = is_npu()
except RuntimeError:
    npu = False  # half-installed torch_npu; fall back to CUDA/CPU

When it happens

Trigger: Importing torch_npu (directly or via a build with it bundled) on a machine with no Ascend NPU, or with NPU devices not visible (ASCEND_RT_VISIBLE_DEVICES empty/invalid).

Common situations: Running a torch_npu-enabled build on a plain CUDA/CPU box; mis-set ASCEND_RT_VISIBLE_DEVICES; driver/cdev permissions hiding NPUs in containers.

Related errors

• Can not import FA3 in sgl_kernel. Please check your installa
• op {op!r} has no backend usable on device {platform.device.v
• GenerativeModel
• {name} is required for NPU packed attention
• {name} must be a 1D int32 or int64 tensor

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7d0bedc8505661e4. Report an issue: GitHub.