sgl-project/sglang · error · RuntimeError

Failed to get server info. {error_data['error']['message']}

Error message

Failed to get server info. {error_data['error']['message']}

What it means

Raised by RuntimeEndpoint.get_server_info when the SGLang server's /server_info endpoint returns a non-200 status. The response body is expected to contain {'error': {'message': ...}} and that message is surfaced in the exception. It means the HTTP request reached something, but the server rejected it or is unhealthy.

Source

Thrown at python/sglang/lang/backend/runtime_endpoint.py:554

        )
        return json.dumps(response.json())

    def encode(
        self,
        prompt: Union[str, List[str], List[Dict], List[List[Dict]]],
    ):
        json_data = {"text": prompt}
        response = requests.post(self.url + "/encode", json=json_data)
        return json.dumps(response.json())

    async def get_server_info(self):
        async with aiohttp.ClientSession() as session:
            async with session.get(f"{self.url}/server_info") as response:
                if response.status == 200:
                    return await response.json()
                else:
                    error_data = await response.json()
                    raise RuntimeError(
                        f"Failed to get server info. {error_data['error']['message']}"
                    )

    def __del__(self):
        self.shutdown()

View on GitHub (pinned to 0132848349)

Solutions

  1. Verify the server is up: curl http://<host>:<port>/health and curl http://<host>:<port>/server_info
  2. Fix the URL passed to RuntimeEndpoint (correct host/port)
  3. Wait for server readiness (health check loop) before calling get_server_info
  4. If behind a proxy/auth, ensure it forwards /server_info unmodified or bypass it

Example fix

// before
endpoint = RuntimeEndpoint("http://localhost:30000")
info = endpoint.get_server_info()
// after
endpoint = RuntimeEndpoint("http://localhost:30000")
# wait until healthy
import requests, time
for _ in range(60):
    try:
        if requests.get("http://localhost:30000/health", timeout=2).status_code == 200:
            break
    except Exception:
        pass
    time.sleep(1)
info = endpoint.get_server_info()
Defensive patterns

Strategy: retry

Validate before calling

import requests
def server_ready(url, tries=30, delay=1):
    for _ in range(tries):
        try:
            if requests.get(f"{url}/server_info", timeout=2).status_code == 200:
                return True
        except Exception:
            pass
        time.sleep(delay)
    return False
assert server_ready("http://localhost:30000")

Try / catch

try:
    info = endpoint.get_server_info()
except (RuntimeError, KeyError) as e:
    # KeyError if error body lacks 'error'/'message'
    logger.error("server_info failed: %s", e)
    raise

Prevention

When it happens

Trigger: Calling endpoint.get_server_info() (directly or via lang frontends that fetch server info) when the server at self.url is down/misrouted, the URL points to a non-SGLang service, the server is still starting up, or a proxy/gateway returns an error JSON body. Also triggers if the error body lacks the 'error'/'message' keys, producing a KeyError instead.

Common situations: Wrong --port or host in the RuntimeEndpoint URL; server not yet ready when the client connects; hitting an ingress that returns its own error JSON; version mismatch where /server_info was renamed or requires auth.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1200d03304253de3. Report an issue: GitHub.