{"record":{"id":"7d8874b236df2771","repo":"sgl-project/sglang","slug":"failed-to-setup-mooncake-embedding-store-ret-cod","errorCode":null,"errorMessage":"Failed to setup Mooncake Embedding Store: {ret_code}","messagePattern":"Failed to setup Mooncake Embedding Store: (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/mem_cache/storage/mooncake_store/mooncake_embedding_store.py","lineNumber":46,"sourceCode":"                self.config.local_hostname,\n                self.config.metadata_server,\n                self.config.global_segment_size,\n                16 * 1024 * 1024,  # Internal local buffer size\n                self.config.protocol,\n                self.config.device_name,\n                self.config.master_server_address,\n                **setup_kwargs,\n            )\n        except TypeError as e:\n            if \"tenant_id\" in setup_kwargs and \"tenant_id\" in str(e):\n                raise RuntimeError(\n                    \"The installed Mooncake version does not support tenant_id \"\n                    \"in MooncakeDistributedStore.setup(). Please upgrade \"\n                    \"Mooncake to use non-default Mooncake tenants with SGLang.\"\n                ) from e\n            raise\n        if ret_code != 0:\n            raise RuntimeError(f\"Failed to setup Mooncake Embedding Store: {ret_code}\")\n\n        logger.info(\"Mooncake Embedding Store initialized successfully.\")\n\n    def get_key(self, mm_hash: str) -> str:\n        return f\"emb_{mm_hash}\"\n\n    def batch_get(\n        self, hashes: List[str], ptrs: List[int], sizes: List[int]\n    ) -> List[bool]:\n        keys = [self.get_key(h) for h in hashes]\n        results = self.store.batch_get_into(keys, ptrs, sizes)\n        return [res > 0 for res in results]\n\n    def batch_put(\n        self, hashes: List[str], ptrs: List[int], sizes: List[int]\n    ) -> List[bool]:\n        keys = [self.get_key(h) for h in hashes]\n        exists = self.store.batch_is_exist(keys)","sourceCodeStart":28,"sourceCodeEnd":64,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/storage/mooncake_store/mooncake_embedding_store.py#L28-L64","documentation":"MooncakeDistributedStore.setup() for the embedding store returned a non-zero return code, meaning the Transfer Engine failed to initialize (master unreachable, bad device/protocol config, etc.). The numeric ret_code is embedded in the message.","triggerScenarios":"MooncakeEmbeddingStore init where the master server address is wrong/unreachable, transfer engine protocol (rdma/tcp) unsupported on the NIC, or metadata server (e.g. P2PHANDSHAKE) misconfigured.","commonSituations":"Wrong master_server_address:PORT in the mooncake JSON config; RDMA devices absent while protocol=rdma; mooncake master not started before SGLang.","solutions":["Verify the mooncake master/transfer-engine metaserver is running at the configured address","Check config file's master_server_address/client_server_address","Use protocol=tcp if no RDMA NIC","Inspect mooncake server logs for the ret_code meaning; retry after fixing"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"def master_reachable(addr: str) -> bool:\n    host, _, port = addr.rpartition(':')\n    import socket\n    try:\n        with socket.create_connection((host, int(port)), timeout=3):\n            return True\n    except OSError:\n        return False","typeGuard":null,"tryCatchPattern":"try:\n    s = MooncakeEmbeddingStore(config)\nexcept RuntimeError as e:\n    if 'Failed to setup Mooncake Embedding Store' in str(e):\n        # log config (redacted), verify master address, retry after backend fix\n        raise","preventionTips":["Pre-flight TCP check on the mooncake master address","Keep mooncake config under version control and validate keys","Start mooncake services before SGLang"],"tags":["mooncake","setup","transfer-engine","config"],"backgroundTag":"remote-store-setup-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}