sgl-project/sglang · critical · RuntimeError

Out-of-tree serve backends cannot replace reserved or built-

Error message

Out-of-tree serve backends cannot replace reserved or built-in backends: {names}

What it means

init_custom_qr (quick allreduce, used for small messages) validates world size: >8 GPUs, exactly 6 GPUs, and odd counts are all unsupported by the quick-reduce kernels.

Source

Thrown at python/sglang/cli/serve_backends.py:100

    def __init__(self, builtins: Mapping[str, ServeBackend]) -> None:
        invalid_builtin_names = set(builtins) & RESERVED_SERVE_BACKEND_NAMES
        if invalid_builtin_names:
            names = ", ".join(sorted(invalid_builtin_names))
            raise ValueError(f"Reserved serve backend names cannot be used: {names}")

        self._builtins = dict(builtins)
        self._entry_points = self._discover_entry_points()
        self._loaded: dict[str, RegisteredServeBackend] = {
            name: RegisteredServeBackend(name=name, backend=backend)
            for name, backend in self._builtins.items()
        }

        reserved = (set(self._builtins) | RESERVED_SERVE_BACKEND_NAMES) & set(
            self._entry_points
        )
        if reserved:
            names = ", ".join(sorted(reserved))
            raise RuntimeError(
                "Out-of-tree serve backends cannot replace reserved or built-in "
                f"backends: {names}"
            )

    @staticmethod
    def _discover_entry_points() -> dict[str, list[EntryPoint]]:
        discovered: dict[str, list[EntryPoint]] = {}
        for entry_point in entry_points(group=SERVE_BACKENDS_GROUP):
            discovered.setdefault(entry_point.name, []).append(entry_point)
        return discovered

    @property
    def available_names(self) -> tuple[str, ...]:
        """Return backend names without importing out-of-tree packages."""

        external_names = sorted(set(self._entry_points) - set(self._builtins))
        return (*self._builtins, *external_names)

View on GitHub (pinned to 0132848349)

Solutions

1. Use TP in {2,4,8} for quick allreduce
2. Disable quick allreduce (fall back to standard custom allreduce or RCCL) for unsupported sizes
3. Verify CUDA/HIP_VISIBLE_DEVICES yields a supported count

Example fix

# before
python -m sglang.launch_server --tp 6 ...  # quick allreduce enabled
# after
python -m sglang.launch_server --tp 4 ...  # or disable quick allreduce

ws = torch.distributed.get_world_size()
assert ws in (2,4,8), f'quick allreduce supports 2/4/8 GPUs, got {ws}'

When it happens

Trigger: Calling init_custom_qr with world_size > 8, == 6, odd, or with rank out of range; e.g. TP=6 or TP=12 on the quick allreduce path.

Common situations: Running TP=6 (common 3-node x2 or 6-GPU setups) with quick allreduce enabled; TP>8 topologies on ROCm; odd visible GPU counts from bad device masks.

Related errors

• delta payload size mismatch: expected ${expectedSize}, got $
• This browser does not support worker image decoding
• Generate subcommand is not yet supported for model: {model_p
• Error: --model-type requires a non-empty value.
• Missing previous frame for delta payload

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/885bc071e3ccb882. Report an issue: GitHub.