sgl-project/sglang · error · ValueError

batching config must be a {'schema_version': 1, 'rules': [..

Error message

batching config must be a {'schema_version': 1, 'rules': [...]} object, a list of rules, or a mapping keyed by model|resolution

What it means

_config_entries accepts only three payload shapes: an object with a 'rules' list, a top-level list of rule dicts, or a mapping keyed by model|resolution strings. Anything else (numbers, strings, objects without 'rules') raises this ValueError.

Source

Thrown at python/sglang/multimodal_gen/runtime/managers/dynamic_batch_admission.py:302

        raise ValueError("batching config schema_version must be 1")
    if isinstance(payload, dict) and isinstance(payload.get("rules"), list):
        return payload["rules"]
    if isinstance(payload, list):
        return payload
    if isinstance(payload, dict):
        entries: list[dict[str, Any]] = []
        for key, value in payload.items():
            if key == "schema_version" or not isinstance(value, dict):
                continue
            model, _sep, resolution = key.partition("|")
            entry = dict(value)
            if model:
                entry.setdefault("model", model)
            if resolution:
                entry.setdefault("resolution", resolution)
            entries.append(entry)
        return entries
    raise ValueError(
        "batching config must be a {'schema_version': 1, 'rules': [...]} object, "
        "a list of rules, or a mapping keyed by model|resolution"
    )


def _validate_rule_keys(data: dict[str, Any], *, source: str) -> None:
    unknown = sorted(set(data) - _BATCHING_RULE_KEYS)
    if not unknown:
        return

    hints = []
    for key in unknown:
        matches = get_close_matches(key, _BATCHING_RULE_KEYS, n=1)
        if matches:
            hints.append(f"{key!r} (did you mean {matches[0]!r}?)")
        else:
            hints.append(repr(key))
    raise ValueError(

View on GitHub (pinned to 0132848349)

Solutions

1. Restructure the file to {"schema_version": 1, "rules": [ ...rule dicts... ]}
2. Verify the file at the batching-config path is actually a batching config and valid JSON
3. For the mapping form, key entries by 'model', 'resolution', or 'model|resolution' strings

Example fix

// before
{"batching": {"qwen-image": {"max_batch_size": 4}}}
// after
{"schema_version": 1, "rules": [{"model": "qwen-image", "max_batch_size": 4}]}

import json
p = json.load(open(path))
ok = isinstance(p, list) or (isinstance(p, dict) and isinstance(p.get("rules"), list))
if not ok:
    raise SystemExit("config root must be a rules object or list")

def is_valid_batching_payload(p) -> bool:
    return isinstance(p, list) or (
        isinstance(p, dict) and isinstance(p.get("rules"), list)
    )

When it happens

Trigger: Loading a JSON file whose root is a string/int, or a dict that is not key-able as model|resolution and has no 'rules' list (e.g. {"batch": {...}}).

Common situations: Wrong file passed to the config path (e.g. a server-args JSON); malformed YAML-to-JSON conversion; mapping keys containing '|' in an unexpected format so the parser takes a different branch and falls through.

Related errors

• batching config rule from {source} must be an object, got {t
• batching config rule requires max_batch_size
• batching config rule cannot set both model and model_contain
• batching config rule requires model or model_contains
• batching config rule max_batch_size must be >= 1

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b771d3ed1eea6a58. Report an issue: GitHub.