sgl-project/sglang · error · RuntimeError

MooncakeStore with standalone_storage=True requires Mooncake

Error message

MooncakeStore with standalone_storage=True requires MooncakeHostTensorAllocator. Please set standalone_storage=False or upgrade Mooncake by 'pip install mooncake-transfer-engine --upgrade'.

What it means

standalone_storage=True requires the host pool to be allocated by MooncakeHostTensorAllocator (memory owned/registered by the transfer engine via setup_dummy), but the pool's allocator is a different type, so initialization aborts.

Source

Thrown at python/sglang/srt/mem_cache/storage/mooncake_store/mooncake_store.py:460

            if device_name and device_name.strip().startswith("{"):
                try:
                    device_config = json.loads(device_name)
                    if storage_config and hasattr(storage_config, "tp_rank"):
                        tp_rank = storage_config.tp_rank
                        # Try both integer and string keys since JSON parsing may convert keys
                        device_name = device_config.get(tp_rank, "")
                        if not device_name:
                            device_name = device_config.get(str(tp_rank), "")
                    else:
                        device_name = ""
                except (json.JSONDecodeError, AttributeError):
                    logger.warning(
                        f"Failed to parse device_name as JSON: {device_name}"
                    )
                    device_name = ""
            if self.config.standalone_storage:
                if not isinstance(mem_pool.allocator, MooncakeHostTensorAllocator):
                    raise RuntimeError(
                        "MooncakeStore with standalone_storage=True requires MooncakeHostTensorAllocator. "
                        "Please set standalone_storage=False "
                        "or upgrade Mooncake by 'pip install mooncake-transfer-engine --upgrade'."
                    )
                required_bytes = self._standalone_required_bytes(mem_pool)
                ret_code = self.store.setup_dummy(
                    required_bytes,
                    DEFAULT_LOCAL_BUFFER_SIZE,  # Zero copy interface does not need local buffer
                    self.config.client_server_address,
                )
            else:
                try:
                    from sglang.srt.distributed.parallel_state import (
                        get_mooncake_transfer_engine,
                    )

                    self._shared_mooncake_transfer_engine = (
                        get_mooncake_transfer_engine()

View on GitHub (pinned to 0132848349)

Solutions

1. Set standalone_storage=False
2. Or configure the host pool to use MooncakeHostTensorAllocator (correct server flag/env)
3. Or pip install mooncake-transfer-engine --upgrade so the required allocator path is available

assert not cfg.standalone_storage or isinstance(pool.allocator, MooncakeHostTensorAllocator), \
    'standalone_storage=True needs MooncakeHostTensorAllocator'

def standalone_ok(pool, standalone: bool) -> bool:
    return (not standalone) or isinstance(
        getattr(pool, 'allocator', None), MooncakeHostTensorAllocator
    )

try:
    ms = MooncakeStore(...)
except RuntimeError as e:
    if 'standalone_storage=True requires' in str(e):
        cfg.standalone_storage = False
        ms = MooncakeStore(cfg)  # fallback mode
    else:
        raise

When it happens

Trigger: Configuring MooncakeStore with standalone_storage=True while the host KV pool uses the default (mmap/other) allocator instead of MooncakeHostTensorAllocator.

Common situations: Enabling the standalone-storage option on an older Mooncake/SGLang combination; forgetting to set the allocator type in server args when enabling standalone storage.

Related errors

• File {normalized_input} does not exist.
• Failed to parse JSON content from file {normalized_input}
• Failed to read JSON file {normalized_input}: {exc}
• Invalid JSON mapping: {normalized_input}
• Invalid format: expected a mapping from GPU id to IB device

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b82f399115d2fbda. Report an issue: GitHub.