sgl-project/sglang · error · ValueError

batching config {source} does not contain any rules

Error message

batching config {source} does not contain any rules

What it means

load_batching_config parses the JSON file into rule entries and requires at least one rule; an empty rules list (or an empty mapping form) produces this ValueError during DynamicBatchAdmission __init__.

Source

Thrown at python/sglang/multimodal_gen/runtime/managers/dynamic_batch_admission.py:278

    def _get_device_memory_gb(gpu_id: int) -> float | None:
        try:
            return current_platform.get_device_total_memory(gpu_id) / BYTES_PER_GB
        except Exception:
            return None


def load_batching_config(path: str | None) -> list[BatchingRule]:
    if path is None:
        return []

    with open(path, encoding="utf-8") as f:
        payload = json.load(f)

    source = os.path.abspath(path)
    entries = _config_entries(payload)
    rules = [BatchingRule.from_dict(entry, source=source) for entry in entries]
    if not rules:
        raise ValueError(f"batching config {source} does not contain any rules")
    return rules


def _config_entries(payload: Any) -> list[dict[str, Any]]:
    if isinstance(payload, dict) and payload.get("schema_version") not in (None, 1):
        raise ValueError("batching config schema_version must be 1")
    if isinstance(payload, dict) and isinstance(payload.get("rules"), list):
        return payload["rules"]
    if isinstance(payload, list):
        return payload
    if isinstance(payload, dict):
        entries: list[dict[str, Any]] = []
        for key, value in payload.items():
            if key == "schema_version" or not isinstance(value, dict):
                continue
            model, _sep, resolution = key.partition("|")
            entry = dict(value)
            if model:

View on GitHub (pinned to 0132848349)

Solutions

1. Add at least one valid rule to the config file
2. If batching config is not wanted, unset the config path / CLI flag so the manager uses defaults instead of an empty file

Example fix

// before
{"schema_version": 1, "rules": []}
// after
{"schema_version": 1, "rules": [{"model_contains": "", "max_batch_size": 8}]}

import json
payload = json.load(open(path))
entries = payload.get("rules") if isinstance(payload, dict) else payload
if not entries:
    raise SystemExit("batching config has no rules; delete the file or add rules")

When it happens

Trigger: Pointing the batching config path at a JSON file whose top-level is {"rules": []}, [], or a mapping with no entries.

Common situations: Placeholder config created empty pending tuning; filtering script emitted zero rules; wrong file path resolves to an intentionally-empty shared config.

Related errors

• batching config rule from {source} must be an object, got {t
• batching config rule requires max_batch_size
• batching config rule cannot set both model and model_contain
• batching config rule requires model or model_contains
• batching config rule max_batch_size must be >= 1

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/08f2933244a3a812. Report an issue: GitHub.