sgl-project/sglang · critical · RuntimeError

mori.umbp is not available. Build mori with BUILD_UMBP=ON or

Error message

mori.umbp is not available. Build mori with BUILD_UMBP=ON or fall back to the default torch host allocator.

What it means

UMBP host allocator requires the native mori.umbp module; importing it failed, so the allocator cannot run. mori must be built with BUILD_UMBP=ON or you must use the default torch host allocator.

Source

Thrown at python/sglang/srt/mem_cache/storage/umbp/umbp_host_allocator.py:34

    if raw is None:
        return default
    return raw.strip().lower() in ("1", "true", "yes", "on")


def _int_env(name: str, default: int) -> int:
    raw = os.getenv(name)
    return int(raw) if raw is not None and raw != "" else default


class UMBPHostTensorAllocator(HostTensorAllocator):
    """Allocate the HiCache L2 host tensor from mori's UMBPHostMemAllocator."""

    def __init__(self) -> None:
        super().__init__()
        try:
            import mori.umbp as umbp_mod
        except ImportError as exc:
            raise RuntimeError(
                "mori.umbp is not available. Build mori with BUILD_UMBP=ON "
                "or fall back to the default torch host allocator."
            ) from exc

        self._mod = umbp_mod
        self._allocator = umbp_mod.UMBPHostMemAllocator()

        self._use_hugepage = _bool_env("SGLANG_HICACHE_HOST_HUGEPAGE", True)
        self._hugepage_size = _int_env(
            "SGLANG_HICACHE_HOST_HUGEPAGE_SIZE", 2 * 1024 * 1024
        )
        self._numa_node = _int_env("SGLANG_HICACHE_HOST_NUMA_NODE", -1)
        self._prefault = _bool_env("SGLANG_HICACHE_HOST_PREFAULT", True)
        self._handles: Dict[int, Any] = {}

    def allocate(
        self, dims: tuple, dtype: torch.dtype, device: str = "cpu"
    ) -> torch.Tensor:

View on GitHub (pinned to 0132848349)

Solutions

Install/rebuild mori with BUILD_UMBP=ON (pip install with the right build flags or use the provided wheel)
If you don't need UMBP hugepage/NUMA features, fall back to the default torch host allocator
Verify 'python -c "import mori.umbp"' succeeds in the serving environment

Example fix

# before
alloc = UMBPHostTensorAllocator()
# after
try:
    import mori.umbp  # noqa
    alloc = UMBPHostTensorAllocator()
except ImportError:
    alloc = None  # fall back to default torch host allocator

Defensive patterns

Strategy: fallback

Validate before calling

try:
    import mori.umbp  # noqa: F401
    UMBP_OK = True
except ImportError:
    UMBP_OK = False
if not UMBP_OK:
    # configure default torch host allocator instead
    pass

Type guard

def umbp_available() -> bool:
    try:
        import mori.umbp  # noqa: F401
        return True
    except ImportError:
        return False

Try / catch

try:
    alloc = UMBPHostTensorAllocator()
except RuntimeError:
    logger.warning('mori.umbp missing; using default host allocator')
    alloc = None  # default allocator path

Prevention

Add a startup capability probe for mori.umbp before selecting the UMBP backend
Document the BUILD_UMBP=ON requirement in deployment docs
Containerize the serving image with a prebuilt mori wheel including umbp

When it happens

Trigger: Instantiating UMBPHostTensorAllocator when mori is not installed or was built without the UMBP extension (import mori.umbp raises ImportError).

Common situations: Using the UMBP storage backend (--storage-backend umbp or similar) with a stock/older mori wheel that lacks umbp; missing LD_LIBRARY_PATH for the shared object.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/70e7c933f85e9418. Report an issue: GitHub.