sgl-project/sglang · critical · RuntimeError

Failed to load serve backend {name!r} from {self._entry_poin

Error message

Failed to load serve backend {name!r} from {self._entry_point_provider(entry_point)}: {exc}

What it means

Raised when the backend's entry-point factory fails to load or call — the entry point resolves but entry_point.load() or factory() raises, or the resolved object is not callable. The RuntimeError chains the original exception and names the provider distribution, distinguishing load failures from unknown or ambiguous names.

Source

Thrown at python/sglang/cli/serve_backends.py:149

        if len(candidates) > 1:
            providers = ", ".join(
                sorted(
                    self._entry_point_provider(candidate) for candidate in candidates
                )
            )
            raise RuntimeError(
                f"Multiple distributions register serve backend {name!r}: "
                f"{providers}. Uninstall one provider or choose another backend name."
            )

        entry_point = candidates[0]
        try:
            factory = entry_point.load()
            if not callable(factory):
                raise TypeError("the entry point must resolve to a callable factory")
            backend = factory()
        except Exception as exc:
            raise RuntimeError(
                f"Failed to load serve backend {name!r} from "
                f"{self._entry_point_provider(entry_point)}: {exc}"
            ) from exc

        if not isinstance(backend, ServeBackend):
            raise TypeError(
                f"Serve backend {name!r} factory returned {type(backend).__name__}; "
                "expected sglang.cli.serve_backends.ServeBackend."
            )
        if backend.api_version != SERVE_BACKEND_API_VERSION:
            raise RuntimeError(
                f"Serve backend {name!r} uses API version {backend.api_version}; "
                f"this SGLang release requires version {SERVE_BACKEND_API_VERSION}."
            )

        registered = RegisteredServeBackend(
            name=name,
            backend=backend,

View on GitHub (pinned to 0132848349)

Solutions

Read the chained `: {exc}` part — the underlying exception is the real cause; fix that first (install missing deps, fix the plugin import).
Upgrade or reinstall the provider package to match your sglang version.
Ensure the entry point targets a zero-arg callable factory (function or class), not an instance or module.
Uninstall the broken plugin and use the built-in 'llm' backend if it fits.

Example fix

# pyproject.toml — before: points at a non-callable
[project.entry-points.'sglang.serve_backends']
mybackend = "my_pkg.backend:BACKEND_INSTANCE"
# after
mybackend = "my_pkg.backend:make_backend"

Defensive patterns

Strategy: try-catch

Validate before calling

from importlib.metadata import entry_points
ep = next(iter(entry_points(group="sglang.serve_backends", name=name)), None)
if ep is None:
    raise SystemExit("backend not installed")
try:
    assert callable(ep.load()), "factory must be callable"
except Exception:
    raise SystemExit(f"backend {name} is broken; see chained error")

Try / catch

try:
    backend = registry.get(name)
except RuntimeError as e:
    if "Failed to load serve backend" in str(e) and e.__cause__:
        log.exception("plugin load failed", exc_info=e.__cause__)

Prevention

Smoke-test plugin imports (`import plugin_pkg`) in a startup healthcheck.
Keep plugin dependency pins in sync with sglang releases.
Fail fast in the plugin factory with clear errors so the chained message is actionable.

When it happens

Trigger: A plugin whose factory module has an ImportError (missing dependency); a factory that raises during instantiation (bad config, missing env var); an entry point pointing at a non-callable attribute; calling registry.get(name) for such a backend.

Common situations: Plugin package missing a runtime dependency in the user's env; plugin built against an older sglang internal API that moved; entry point pointing to a class instead of a factory function.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/05dba990cdae54bb. Report an issue: GitHub.