sgl-project/sglang · error · RuntimeError

engine_info_bootstrap_port {bootstrap_port} is already in us

Error message

engine_info_bootstrap_port {bootstrap_port} is already in use. When running multiple instances on the same node, each instance must use a different --engine-info-bootstrap-port.

What it means

When the remote-instance weight loader transfer engine is enabled on node_rank 0, the engine starts an EngineInfoBootstrapServer on --engine-info-bootstrap-port. That port must be free; if another process (typically another SGLang instance on the same host) already binds it, startup fails with RuntimeError.

Source

Thrown at python/sglang/srt/entrypoints/engine.py:1089

        # Nothing below has spawned yet, so a failure here leaves a record the
        # caller can hand back -- but only once the publication goes with it:
        # the validation stage writes through late resolution, which refuses a
        # record that is already published.
        try:
            # Allocate ports for inter-process communications
            if port_args is None:
                port_args = PortArgs.init_new(server_args)
            logger.info(f"server_args={server_args.resolved_dict()}")

            # Start the engine info bootstrap server if per-rank info is needed.
            engine_info_bootstrap_server = None
            if (
                get_model().remote_instance_weight_loader_start_seed_via_transfer_engine
                and server_args.node_rank == 0
            ):
                bootstrap_port = server_args.engine_info_bootstrap_port
                if not is_port_available(bootstrap_port):
                    raise RuntimeError(
                        f"engine_info_bootstrap_port {bootstrap_port} is already in use. "
                        f"When running multiple instances on the same node, each instance must use a "
                        f"different --engine-info-bootstrap-port."
                    )
                engine_info_bootstrap_server = EngineInfoBootstrapServer(
                    host=server_args.host, port=bootstrap_port
                )

            # Launch daemons (daemon mode only). The handles travel back to the
            # Engine that spawned them; shutdown() reaps from there.
            weight_cache_daemon_procs: List = []
            if server_args.weight_cache_mode == "daemon":
                weight_cache_daemon_procs = cls._launch_weight_cache_daemons(
                    server_args
                )
        except BaseException:
            restore_context(context_before_publish)
            raise

View on GitHub (pinned to 0132848349)

Solutions

1. Give each instance a distinct --engine-info-bootstrap-port (e.g. 8100, 8101, ...).
2. Kill the stale process holding the port (check with ss -ltnp | grep <port>) and relaunch.
3. If the feature isn't needed for this instance, disable the remote-instance weight loader path so no bootstrap server is started.

Example fix

# before (both instances)
python -m sglang.launch_server --model ... # same default bootstrap port -> second fails

# after (instance 2)
python -m sglang.launch_server --model ... --engine-info-bootstrap-port 8102

import socket
def port_free(p):
    s = socket.socket(); s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    try:
        s.bind(("0.0.0.0", p)); return True
    except OSError:
        return False
    finally:
        s.close()
assert port_free(engine_info_bootstrap_port)

try:
    engine = sgl.Engine(**kwargs)
except RuntimeError as e:
    if "engine_info_bootstrap_port" in str(e):
        kwargs["engine_info_bootstrap_port"] += 1
        engine = sgl.Engine(**kwargs)
    else:
        raise

When it happens

Trigger: Two or more SGLang instances with remote-instance weight loading running on one node — the second one finds engine_info_bootstrap_port occupied and raises at _launch_subprocesses time.

Common situations: Co-located instances for throughput testing; k8s pods sharing host network; a previous instance that didn't fully shut down still holding the port; fixed default port used by all instances.

Related errors

• SGLANG_RUST_SERVER serves the PD KV bootstrap registry on th
• SGLANG_RUST_SERVER is not supported with the offline Engine
• Multi-node weight cache daemons (nnodes > 1) require --dist-
• Weight cache daemon for pp_rank={pp_rank} tp_rank={tp_rank}
• Weight cache daemon (pid={p.pid}) exited prematurely with co

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fe653b42b58073d5. Report an issue: GitHub.