sgl-project/sglang · critical · RuntimeError

Failed to setup Mooncake Embedding Store: {ret_code}

Error message

Failed to setup Mooncake Embedding Store: {ret_code}

What it means

MooncakeDistributedStore.setup() for the embedding store returned a non-zero return code, meaning the Transfer Engine failed to initialize (master unreachable, bad device/protocol config, etc.). The numeric ret_code is embedded in the message.

Source

Thrown at python/sglang/srt/mem_cache/storage/mooncake_store/mooncake_embedding_store.py:46

                self.config.local_hostname,
                self.config.metadata_server,
                self.config.global_segment_size,
                16 * 1024 * 1024,  # Internal local buffer size
                self.config.protocol,
                self.config.device_name,
                self.config.master_server_address,
                **setup_kwargs,
            )
        except TypeError as e:
            if "tenant_id" in setup_kwargs and "tenant_id" in str(e):
                raise RuntimeError(
                    "The installed Mooncake version does not support tenant_id "
                    "in MooncakeDistributedStore.setup(). Please upgrade "
                    "Mooncake to use non-default Mooncake tenants with SGLang."
                ) from e
            raise
        if ret_code != 0:
            raise RuntimeError(f"Failed to setup Mooncake Embedding Store: {ret_code}")

        logger.info("Mooncake Embedding Store initialized successfully.")

    def get_key(self, mm_hash: str) -> str:
        return f"emb_{mm_hash}"

    def batch_get(
        self, hashes: List[str], ptrs: List[int], sizes: List[int]
    ) -> List[bool]:
        keys = [self.get_key(h) for h in hashes]
        results = self.store.batch_get_into(keys, ptrs, sizes)
        return [res > 0 for res in results]

    def batch_put(
        self, hashes: List[str], ptrs: List[int], sizes: List[int]
    ) -> List[bool]:
        keys = [self.get_key(h) for h in hashes]
        exists = self.store.batch_is_exist(keys)

View on GitHub (pinned to 0132848349)

Solutions

1. Verify the mooncake master/transfer-engine metaserver is running at the configured address
2. Check config file's master_server_address/client_server_address
3. Use protocol=tcp if no RDMA NIC
4. Inspect mooncake server logs for the ret_code meaning; retry after fixing

def master_reachable(addr: str) -> bool:
    host, _, port = addr.rpartition(':')
    import socket
    try:
        with socket.create_connection((host, int(port)), timeout=3):
            return True
    except OSError:
        return False

try:
    s = MooncakeEmbeddingStore(config)
except RuntimeError as e:
    if 'Failed to setup Mooncake Embedding Store' in str(e):
        # log config (redacted), verify master address, retry after backend fix
        raise

When it happens

Trigger: MooncakeEmbeddingStore init where the master server address is wrong/unreachable, transfer engine protocol (rdma/tcp) unsupported on the NIC, or metadata server (e.g. P2PHANDSHAKE) misconfigured.

Common situations: Wrong master_server_address:PORT in the mooncake JSON config; RDMA devices absent while protocol=rdma; mooncake master not started before SGLang.

Related errors

• Failed to setup Mooncake store, error code: {ret_code}
• File {normalized_input} does not exist.
• Failed to parse JSON content from file {normalized_input}
• Failed to read JSON file {normalized_input}: {exc}
• Invalid JSON mapping: {normalized_input}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7d8874b236df2771. Report an issue: GitHub.