sgl-project/sglang · error · ValueError

SGLANG_RUST_SERVER is not supported with the offline Engine

Error message

SGLANG_RUST_SERVER is not supported with the offline Engine API; it only replaces the HTTP server path (`sglang serve`). Unset SGLANG_RUST_SERVER to use sgl.Engine.

What it means

sgl.Engine (the offline Python Engine API) does not support SGLANG_RUST_SERVER. That env var only swaps the HTTP server frontend used by `sglang serve` for a Rust implementation; the offline Engine spawns schedulers directly and has no HTTP path to replace. Constructing sgl.Engine with the env var set raises ValueError immediately after server_args resolution.

Source

Thrown at python/sglang/srt/entrypoints/engine.py:258

        # so hooks on ServerArgs.__post_init__ fire correctly.
        load_plugins()

        # Parse server_args
        if "server_args" in kwargs:
            # Directly load server_args
            server_args = kwargs["server_args"]
        else:
            # Construct server_args from kwargs
            if "log_level" not in kwargs:
                # Do not print logs by default
                kwargs["log_level"] = "error"
            server_args = self.server_args_class(**kwargs)
        self.server_args = server_args
        logger.info(f"server_args={server_args.resolved_dict()}")

        # Rust Server is not supported with the offline Engine API
        if envs.SGLANG_RUST_SERVER.get():
            raise ValueError(
                "SGLANG_RUST_SERVER is not supported with the offline Engine "
                "API; it only replaces the HTTP server path (`sglang serve`). "
                "Unset SGLANG_RUST_SERVER to use sgl.Engine."
            )

        # Pre-initialize tokenizer_manager so the atexit handler in
        # shutdown() won't hit AttributeError.
        self.tokenizer_manager = None

        # Shutdown the subprocesses automatically when the program exits
        atexit.register(self.shutdown)

        # Launch subprocesses
        (
            tokenizer_manager,
            template_manager,
            port_args,
            scheduler_init_result,

View on GitHub (pinned to 0132848349)

Solutions

1. Unset the variable before creating the engine: `env -u SGLANG_RUST_SERVER python script.py` or `SGLANG_RUST_SERVER= python script.py`.
2. In Python, run `os.environ.pop("SGLANG_RUST_SERVER", None)` before instantiating sgl.Engine.
3. If you actually want the Rust server path, use `sglang serve` (the HTTP entrypoint) instead of sgl.Engine.
4. Audit shell profiles, Dockerfiles, and CI env blocks for a globally exported SGLANG_RUST_SERVER.

Example fix

# before
import sglang as sgl
engine = sgl.Engine(model_path="...")  # ValueError: SGLANG_RUST_SERVER set

# after
import os
os.environ.pop("SGLANG_RUST_SERVER", None)
import sglang as sgl
engine = sgl.Engine(model_path="...")

import os, sglang as sgl
assert not os.environ.get("SGLANG_RUST_SERVER"), (
    "unset SGLANG_RUST_SERVER before using sgl.Engine")
engine = sgl.Engine(...)

try:
    engine = sgl.Engine(**kwargs)
except ValueError as e:
    if "SGLANG_RUST_SERVER" in str(e):
        os.environ.pop("SGLANG_RUST_SERVER", None)
        engine = sgl.Engine(**kwargs)
    else:
        raise

When it happens

Trigger: Calling sgl.Engine(**kwargs) (or srt.Engine) in a process where the SGLANG_RUST_SERVER environment variable is set to a truthy value, e.g. exported earlier for a `sglang serve` run and reused in the same shell/notebook for offline generation.

Common situations: Switching between serving and offline batch inference in the same environment; CI images that bake SGLANG_RUST_SERVER=1 in globally; docker exec sessions inheriting the variable from the server container.

Related errors

• SGLANG_DISAGG_STAGING_BUFFER requires disaggregation_transfe
• Multi-node weight cache daemons (nnodes > 1) require --dist-
• Weight cache daemon for pp_rank={pp_rank} tp_rank={tp_rank}
• Weight cache daemon (pid={p.pid}) exited prematurely with co
• engine_info_bootstrap_port {bootstrap_port} is already in us

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3b5af2b4f3964ff2. Report an issue: GitHub.