sgl-project/sglang · warning · HTTPException

{e}

Error message

{e}

What it means

SGLang's /v1/loads endpoint wraps ValueError from tokenizer_manager.get_loads() into an HTTP 400 with the original message as detail. The underlying ValueError typically signals an invalid include filter or an invalid dp_rank for the data-parallel topology.

Source

Thrown at python/sglang/srt/entrypoints/v1_loads.py:126

        include: Comma-separated sections to include (optional)
                 Options: core, memory, spec, lora, disagg, queues, all
                 Default: all
        format: Response format - 'json' (default) or 'prometheus'

    Returns:
        JSON response with timestamp, version, accelerator metadata, and
        per-DP-rank loads
    """
    include_list = [s.strip() for s in include.split(",")] if include else None

    start = time.perf_counter()
    try:
        load_results = await tokenizer_manager.get_loads(
            include=include_list,
            dp_rank=dp_rank,
        )
    except ValueError as e:
        raise HTTPException(status_code=400, detail=str(e))
    finally:
        mc = getattr(tokenizer_manager, "metrics_collector", None)
        if mc is not None:
            mc.get_loads_duration_seconds.labels(**mc.labels).observe(
                time.perf_counter() - start
            )

    include_set = set(include_list) if include_list else None

    if format == "prometheus":
        return _format_loads_prometheus(load_results, include_set)

    loads = []
    for load in load_results:
        d = load.to_dict(include_set)
        loads.append(d)

    return {

View on GitHub (pinned to 0132848349)

Solutions

1. Read the response detail — it is the raw ValueError text naming the invalid parameter
2. Use include values from the documented enum and dp_rank in [0, dp_size-1]
3. Omit dp_rank to query the aggregated load across DP ranks

Example fix

# before
resp = requests.get(f"{base}/v1/loads", params={"include": "bogus", "dp_rank": 8})
# after
resp = requests.get(f"{base}/v1/loads", params={"include": "queue", "dp_rank": 0})

# validate before calling the API
valid_include = {"queue", "cache", "generation"}  # per server docs
params = {"include": "queue", "dp_rank": 0}
assert set(params.get("include", "").split(",")) <= valid_include

try:
    loads = await client.get_loads(include=include, dp_rank=dp_rank)
except HTTPException as e:  # or requests.HTTPError
    if e.response.status_code == 400:
        detail = e.response.json()["detail"]  # original ValueError message
        fix_params_from(detail)
    raise

When it happens

Trigger: GET /v1/load with include=... containing unknown keys, or dp_rank greater than/equal to the server's DP size (or negative), causing get_loads to raise ValueError, which is rethrown as HTTPException(400, str(e)).

Common situations: Client passing an unsupported metrics include list, guessing dp_rank after changing --dp-size, or stale client code after the include enum changed between SGLang versions.

Related errors

• --enable-metrics requires smg-grpc-servicer ≥ 0.5.3 (the ver
• {finish_reason["message"]}
• SGLANG_LOG_SCHEDULER_STATUS_TARGET is set but --enable-metri
• Unknown serve backend {name!r}. Available values: {available
• Multiple distributions register serve backend {name!r}: {pro

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d9a013ffa3ada9cd. Report an issue: GitHub.