sgl-project/sglang · critical · RuntimeError

mori.umbp is not available. Build mori with BUILD_UMBP=ON…

Error message

mori.umbp is not available. Build mori with BUILD_UMBP=ON or fall back to the default torch host allocator.

What it means

Raised when the UMBP host allocator requests the mori.umbp extension but it is not importable. mori must be built with BUILD_UMBP=ON, or the server should fall back to the default torch host allocator.

Solutions

  1. Install/rebuild mori with BUILD_UMBP=ON (pip install with the right build flags or use the provided wheel)
  2. If you don't need UMBP hugepage/NUMA features, fall back to the default torch host allocator
  3. Verify 'python -c "import mori.umbp"' succeeds in the serving environment

Example fix

# before
alloc = UMBPHostTensorAllocator()
# after
try:
    import mori.umbp  # noqa
    alloc = UMBPHostTensorAllocator()
except ImportError:
    alloc = None  # fall back to default torch host allocator
Defensive patterns

Strategy: fallback

Validate before calling

try:
    import mori.umbp  # noqa: F401
    UMBP_OK = True
except ImportError:
    UMBP_OK = False
if not UMBP_OK:
    # configure default torch host allocator instead
    pass

Type guard

def umbp_available() -> bool:
    try:
        import mori.umbp  # noqa: F401
        return True
    except ImportError:
        return False

Try / catch

try:
    alloc = UMBPHostTensorAllocator()
except RuntimeError:
    logger.warning('mori.umbp missing; using default host allocator')
    alloc = None  # default allocator path

Prevention

When it happens

Trigger: Instantiating UMBPHostTensorAllocator when mori is not installed or was built without the UMBP extension (import mori.umbp raises ImportError).

Common situations: Using the UMBP storage backend (--storage-backend umbp or similar) with a stock/older mori wheel that lacks umbp; missing LD_LIBRARY_PATH for the shared object.

Understand the failure class

Background: "X is not installed. Please install it with pip install Y": missing optional dependency errors — ImportError/ValueError raised when a library's optional extra was never installed — this error's family across 22 libraries.

Related errors


AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/70e7c933f85e9418. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/mem_cache/storage/umbp/umbp_host_allocator.py:34

    if raw is None:
        return default
    return raw.strip().lower() in ("1", "true", "yes", "on")


def _int_env(name: str, default: int) -> int:
    raw = os.getenv(name)
    return int(raw) if raw is not None and raw != "" else default


class UMBPHostTensorAllocator(HostTensorAllocator):
    """Allocate the HiCache L2 host tensor from mori's UMBPHostMemAllocator."""

    def __init__(self) -> None:
        super().__init__()
        try:
            import mori.umbp as umbp_mod
        except ImportError as exc:
            raise RuntimeError(
                "mori.umbp is not available. Build mori with BUILD_UMBP=ON "
                "or fall back to the default torch host allocator."
            ) from exc

        self._mod = umbp_mod
        self._allocator = umbp_mod.UMBPHostMemAllocator()

        self._use_hugepage = _bool_env("SGLANG_HICACHE_HOST_HUGEPAGE", True)
        self._hugepage_size = _int_env(
            "SGLANG_HICACHE_HOST_HUGEPAGE_SIZE", 2 * 1024 * 1024
        )
        self._numa_node = _int_env("SGLANG_HICACHE_HOST_NUMA_NODE", -1)
        self._prefault = _bool_env("SGLANG_HICACHE_HOST_PREFAULT", True)
        self._handles: Dict[int, Any] = {}

    def allocate(
        self, dims: tuple, dtype: torch.dtype, device: str = "cpu"
    ) -> torch.Tensor:

View on GitHub (pinned to 0132848349)