sgl-project/sglang · critical · ConnectionError

Could not connect to remote scheduler at {self.server_args.s

Error message

Could not connect to remote scheduler at {self.server_args.scheduler_endpoint} with `local mode` as False. Please ensure the server is running.

What it means

Raised by DiffGenerator._check_remote_scheduler when the synchronous scheduler client's ping() fails while server_args has local mode disabled. The generator expected a remote scheduler at server_args.scheduler_endpoint but could not reach it, so construction via from_server_args aborts with ConnectionError.

Source

Thrown at python/sglang/multimodal_gen/runtime/entrypoints/diffusion_generator.py:200

    ) -> list[mp.Process]:
        """Check if a local server is running; if not, start it and return the process handles."""
        # First, we need a client to test the server. Initialize it temporarily.
        sync_scheduler_client.initialize(self.server_args)

        processes = launch_server(self.server_args, launch_http_server=False)

        return processes

    def _run_client_warmup_if_needed(self) -> None:
        if not should_run_explicit_client_warmup(self.server_args):
            return

        run_sync_client_warmup(self.server_args, sync_scheduler_client.forward)

    def _check_remote_scheduler(self):
        """Check if the remote scheduler is accessible."""
        if not sync_scheduler_client.ping():
            raise ConnectionError(
                f"Could not connect to remote scheduler at "
                f"{self.server_args.scheduler_endpoint} with `local mode` as False. "
                "Please ensure the server is running."
            )
        logger.info(
            f"Successfully connected to remote scheduler at "
            f"{self.server_args.scheduler_endpoint}."
        )

    @staticmethod
    def _resolve_image_paths_per_prompt(
        prompts: list[str], image_paths: str | list[str] | None
    ) -> list[str | list[str] | None]:
        if len(prompts) <= 1:
            return [image_paths]

        if not isinstance(image_paths, list) or len(image_paths) <= 1:
            return [image_paths for _ in prompts]

View on GitHub (pinned to 0132848349)

Solutions

  1. Start the scheduler first (e.g. `sglang serve ...`) and confirm it logs readiness at the same endpoint
  2. Verify scheduler_endpoint host/port in server_args matches where the scheduler is listening; test with a raw TCP/curl connect
  3. If cross-host, make sure the scheduler binds 0.0.0.0 and firewalls allow the port; in k8s wait for the service/readiness probe
  4. If you intended in-process generation, set local mode instead of connecting remotely

Example fix

# before
generator = DiffGenerator.from_server_args(server_args)  # local_mode=False, scheduler not up

# after
# terminal 1: sglang serve --model-path ... --port 30000
generator = DiffGenerator.from_server_args(server_args)  # scheduler reachable
Defensive patterns

Strategy: retry

Validate before calling

import socket
from urllib.parse import urlparse
host, port = parse_endpoint(server_args.scheduler_endpoint)
with socket.create_connection((host, port), timeout=5):
    pass  # scheduler reachable

Try / catch

for attempt in range(5):
    try:
        gen = DiffGenerator.from_server_args(server_args)
        break
    except ConnectionError:
        time.sleep(2 * (attempt + 1))
else:
    raise RuntimeError("scheduler never became reachable")

Prevention

When it happens

Trigger: Instantiating DiffGenerator.from_server_args(server_args) with local_mode=False when no scheduler is listening at scheduler_endpoint — server not started, wrong host/port, firewall blocking, or scheduler crashed after startup.

Common situations: Forgetting to launch the scheduler before the client, wrong port in server_args, scheduler bound to localhost only while client runs in another container, k8s service not ready yet.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1881c473ae7e7506. Report an issue: GitHub.