sgl-project/sglang · error · ValueError

Failed to import sglang.private.private_model_loader

Error message

Failed to import sglang.private.private_model_loader

What it means

get_model_loader dispatches on load_config.load_format; for the PRIVATE format it importlib-imports sglang.private.private_model_loader and instantiates PrivateModelLoader. That module only exists in NVIDIA's internal build, so open-source installs raise ValueError on import failure.

Source

Thrown at python/sglang/srt/model_loader/loader.py:4351

            )
            model_config.quantization = "fp8"

        return QuantizedRLModelLoader(load_config)

    if load_config.load_format == LoadFormat.REMOTE:
        return RemoteModelLoader(load_config)

    if load_config.load_format == LoadFormat.REMOTE_INSTANCE:
        return RemoteInstanceModelLoader(load_config)

    if load_config.load_format == LoadFormat.PRIVATE:
        import importlib

        try:
            module = importlib.import_module("sglang.private.private_model_loader")
            return module.PrivateModelLoader(load_config)
        except ImportError:
            raise ValueError("Failed to import sglang.private.private_model_loader")

    if load_config.load_format == LoadFormat.RUNAI_STREAMER:
        return RunaiModelStreamerLoader(load_config)

    if load_config.load_format == LoadFormat.IPC_CACHE:
        from sglang.srt.weight_cache.ipc_loader import IpcModelLoader
        from sglang.srt.weight_cache.protocol import (
            compute_global_rank,
            get_socket_path,
        )

        if load_config.weight_cache_socket:
            socket_path = load_config.weight_cache_socket
        else:
            from sglang.srt.runtime_context import get_parallel

            ps = get_parallel()
            global_rank = compute_global_rank(ps.tp_size, ps.pp_rank, ps.tp_rank)

View on GitHub (pinned to 0132848349)

Solutions

1. Remove --load-format private / set the load format to a public one (default, auto, modelopt, runai_streamer, ...)
2. If you believe you need it, use the NVIDIA internal sglang build that ships sglang.private
3. Print available formats: python -c "from sglang.srt.model_loader.loader import LoadFormat; print([f.value for f in LoadFormat])"

Example fix

# before
--load-format private
# ValueError: Failed to import sglang.private.private_model_loader

# after
--load-format auto

import importlib.util
if load_format == "private" and importlib.util.find_spec("sglang.private.private_model_loader") is None:
    raise SystemExit("--load-format private requires the internal sglang build")

def private_loader_available() -> bool:
    import importlib.util
    return importlib.util.find_spec("sglang.private.private_model_loader") is not None

try:
    get_model_loader(load_config)
except ValueError as e:
    if "private_model_loader" in str(e):
        load_config.load_format = "auto"  # fall back to public loader
        loader = get_model_loader(load_config)

When it happens

Trigger: Setting --load-format private (or a config resolving to LoadFormat.PRIVATE) in an open-source sglang install where sglang.private.private_model_loader doesn't exist.

Common situations: Copy-pasted launch command meant for an internal deployment; serialized server_args from a private fork replayed on OSS sglang; typo'd load format string silently matched to 'private'.

Related errors

• Failed to load serve backend {name!r} from {self._entry_poin
• Can not import FA3 in sgl_kernel. Please check your installa
• Please install mooncake by following the instructions at htt
• Failed to import 'set_transfer_engine' from 'mooncake.pg'. P
• gRPC mode requires the smg-grpc-servicer package. If not ins

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8888b2fec7ef332d. Report an issue: GitHub.