vllm-project/vllm · critical · RuntimeError

Mooncake Transfer Engine initialization failed.

Error message

Mooncake Transfer Engine initialization failed.

What it means

RuntimeError raised when TransferEngine.initialize(hostname, 'P2PHANDSHAKE', protocol, device_name) returns a nonzero code, meaning the Mooncake transfer engine's native initialization failed. protocol defaults to kv_connector_extra_config 'mooncake_protocol' (fallback 'rdma') and device_name likewise comes from extra config. Nonzero return codes typically mean the transport backend (RDMA/UCX verbs) could not be set up on this host — missing drivers, no RDMA devices, or an unsupported protocol/device string.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/mooncake/mooncake_connector.py:941

        )
        # Create more tasks than workers to keep the thread pool saturated.
        # Tasks can await async events, so a surplus (2x is a robust heuristic)
        # prevents workers from idling.
        self.num_sender_tasks = self.num_sender_workers * 2
        protocol = kv_transfer_config.kv_connector_extra_config.get(  # type: ignore[union-attr]
            "mooncake_protocol", "rdma"
        )
        device_name = kv_transfer_config.kv_connector_extra_config.get(  # type: ignore[union-attr]
            "device_name", ""
        )
        logger.info(
            "The Mooncake Transfer Engine is using %s as its protocol.", protocol
        )
        ret_value = self.engine.initialize(
            self.hostname, "P2PHANDSHAKE", protocol, device_name
        )
        if ret_value != 0:
            raise RuntimeError("Mooncake Transfer Engine initialization failed.")

        self.rpc_port = self.engine.get_rpc_port()

        logger.debug(
            "Mooncake Transfer Engine initialized at %s:%d",
            self.hostname,
            self.rpc_port,
        )

        self._remote_agents: dict[EngineId, dict[int, dict[int, str]]] = {}
        self._pending_bootstrap_queries: dict[str, asyncio.Event] = {}
        self.side_channel_port: int = 0  # we will bind it in register_kv_caches()
        self.engine_id: EngineId = engine_id
        self.tp_rank = get_tensor_model_parallel_rank()
        self.tp_size = get_tensor_model_parallel_world_size()
        self.block_len_per_layer: list[int] = []
        self.kv_block_len_per_layer: list[int] = []
        self.registered_layer_names: list[str] = []

View on GitHub (pinned to c794754062)

Solutions

1. Check the configured mooncake_protocol and device_name in kv_connector_extra_config and confirm the host supports that transport.
2. On hosts without RDMA hardware, use a software transport supported by mooncake (e.g. set protocol to 'ucx' with tcp/posix transport or per mooncake docs) or enable the appropriate mode.
3. Verify RDMA stack health: ibv_devinfo lists devices, ibverbs/ucx libraries installed.
4. In containers, ensure InfiniBand devices and libraries are mounted/installed.

Example fix

# before
extra_config = {"mooncake_protocol": "rdma"}  # host has no RDMA NIC
# after
extra_config = {"mooncake_protocol": "ucx"}  # UCX over TCP/POSIX

import subprocess
def rdma_devices_present() -> bool:
    return subprocess.run(["ibv_devinfo"], capture_output=True).returncode == 0
# choose protocol = 'rdma' only if rdma_devices_present() else 'ucx'

Catch RuntimeError from engine init; inspect protocol/device_name config, switch the transport (e.g. rdma -> ucx) and retry initialization once with the corrected config.

When it happens

Trigger: protocol 'rdma' on a host without RDMA NICs or ibverbs installed; wrong device_name in kv_connector_extra_config; hostname not resolvable/bindable; ucx protocol requested without UCX installed; running in a plain container without /dev/infiniband devices.

Common situations: Testing the Mooncake connector on non-RDMA hardware; Kubernetes pods missing InfiniBand device mounts; typo'd mooncake_protocol or device_name in kv_connector_extra_config; mlx/p2p protocol attempted on unsupported hardware.

Related errors

• Mooncake is not available
• Mooncake batch memory registration failed.
• Mooncake preferred_segment override must be a non-empty stri
• MooncakeStoreConnector does not support: {unsupported}
• block_size ({self.block_size}) must be a multiple of hash_bl

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/83606c95678c5faa. Report an issue: GitHub.