{"record":{"id":"885bc071e3ccb882","repo":"sgl-project/sglang","slug":"out-of-tree-serve-backends-cannot-replace-reserved","errorCode":null,"errorMessage":"Out-of-tree serve backends cannot replace reserved or built-in backends: {names}","messagePattern":"Out-of-tree serve backends cannot replace reserved or built-in backends: (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"python/sglang/cli/serve_backends.py","lineNumber":100,"sourceCode":"    def __init__(self, builtins: Mapping[str, ServeBackend]) -> None:\n        invalid_builtin_names = set(builtins) & RESERVED_SERVE_BACKEND_NAMES\n        if invalid_builtin_names:\n            names = \", \".join(sorted(invalid_builtin_names))\n            raise ValueError(f\"Reserved serve backend names cannot be used: {names}\")\n\n        self._builtins = dict(builtins)\n        self._entry_points = self._discover_entry_points()\n        self._loaded: dict[str, RegisteredServeBackend] = {\n            name: RegisteredServeBackend(name=name, backend=backend)\n            for name, backend in self._builtins.items()\n        }\n\n        reserved = (set(self._builtins) | RESERVED_SERVE_BACKEND_NAMES) & set(\n            self._entry_points\n        )\n        if reserved:\n            names = \", \".join(sorted(reserved))\n            raise RuntimeError(\n                \"Out-of-tree serve backends cannot replace reserved or built-in \"\n                f\"backends: {names}\"\n            )\n\n    @staticmethod\n    def _discover_entry_points() -> dict[str, list[EntryPoint]]:\n        discovered: dict[str, list[EntryPoint]] = {}\n        for entry_point in entry_points(group=SERVE_BACKENDS_GROUP):\n            discovered.setdefault(entry_point.name, []).append(entry_point)\n        return discovered\n\n    @property\n    def available_names(self) -> tuple[str, ...]:\n        \"\"\"Return backend names without importing out-of-tree packages.\"\"\"\n\n        external_names = sorted(set(self._entry_points) - set(self._builtins))\n        return (*self._builtins, *external_names)\n","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/cli/serve_backends.py#L82-L118","documentation":"init_custom_qr (quick allreduce, used for small messages) validates world size: >8 GPUs, exactly 6 GPUs, and odd counts are all unsupported by the quick-reduce kernels.","triggerScenarios":"Calling init_custom_qr with world_size > 8, == 6, odd, or with rank out of range; e.g. TP=6 or TP=12 on the quick allreduce path.","commonSituations":"Running TP=6 (common 3-node x2 or 6-GPU setups) with quick allreduce enabled; TP>8 topologies on ROCm; odd visible GPU counts from bad device masks.","solutions":["Use TP in {2,4,8} for quick allreduce","Disable quick allreduce (fall back to standard custom allreduce or RCCL) for unsupported sizes","Verify CUDA/HIP_VISIBLE_DEVICES yields a supported count"],"exampleFix":"# before\npython -m sglang.launch_server --tp 6 ...  # quick allreduce enabled\n# after\npython -m sglang.launch_server --tp 4 ...  # or disable quick allreduce","handlingStrategy":"validation","validationCode":"ws = torch.distributed.get_world_size()\nassert ws in (2,4,8), f'quick allreduce supports 2/4/8 GPUs, got {ws}'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Avoid TP=6 or >8 when quick allreduce is enabled","Disable quick allreduce for unsupported topologies"],"tags":["rocm","allreduce","quick-allreduce","tensor-parallel"],"backgroundTag":"world-size-unsupported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}