sgl-project/sglang · error · ValueError

Reserved serve backend names cannot be used

Error message

Reserved serve backend names cannot be used: {names}

What it means

Raised during serve-backend registration when an out-of-tree plugin tries to register a backend whose name collides with the reserved backend names. Reserved names are owned by the built-in dispatcher; rename the custom backend to a non-reserved identifier.

Solutions

1. Cast tensors to float32/float16/bfloat16 before the call
2. Exclude such tensors from the deterministic custom path (use RCCL)
3. Note bf16 requires gfx8+ hardware (__HIP_ARCH__ >= 800)

Example fix

# before
deterministic_all_reduce(fa, x_double, out)
# after
deterministic_all_reduce(fa, x_double.float(), out.float())

def det_ar_supported(t: torch.Tensor) -> bool:
    return t.dtype in (torch.float32, torch.float16, torch.bfloat16)

try:
    deterministic_all_reduce(fa, inp, out)
except RuntimeError as e:
    if 'only supports' in str(e):
        deterministic_all_reduce(fa, inp.float(), out.float())
    else:
        raise

When it happens

Trigger: Calling deterministic_all_reduce with dtype float64, integer, or fp8 tensors.

Common situations: Deterministic inference mode enabled for models producing non-fp16/bf16/fp32 tensors; accidental double precision from a cast; fp8 pathways routed into deterministic allreduce.

Related errors

• Error: --model-type requires a non-empty value.
• Error: --model-type requires a value.
• Generate subcommand is not yet supported for model
• Missing previous frame for delta payload
• Previous frame size does not match current delta payload

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a233c64e3ae9c734. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/cli/serve_backends.py:86

@dataclass(frozen=True)
class RegisteredServeBackend:
    """A loaded backend together with its discovery metadata."""

    name: str
    backend: ServeBackend
    distribution: str | None = None


class ServeBackendRegistry:
    """Registry of built-in and installed out-of-tree serve backends."""

    def __init__(self, builtins: Mapping[str, ServeBackend]) -> None:
        invalid_builtin_names = set(builtins) & RESERVED_SERVE_BACKEND_NAMES
        if invalid_builtin_names:
            names = ", ".join(sorted(invalid_builtin_names))
            raise ValueError(f"Reserved serve backend names cannot be used: {names}")

        self._builtins = dict(builtins)
        self._entry_points = self._discover_entry_points()
        self._loaded: dict[str, RegisteredServeBackend] = {
            name: RegisteredServeBackend(name=name, backend=backend)
            for name, backend in self._builtins.items()
        }

        reserved = (set(self._builtins) | RESERVED_SERVE_BACKEND_NAMES) & set(
            self._entry_points
        )
        if reserved:
            names = ", ".join(sorted(reserved))
            raise RuntimeError(
                "Out-of-tree serve backends cannot replace reserved or built-in "
                f"backends: {names}"
            )

View on GitHub (pinned to 0132848349)