sgl-project/sglang · error · ValueError

batching config rule requires max_batch_size

Error message

batching config rule requires max_batch_size

What it means

A batching rule entry in the dynamic batch admission config JSON must include a max_batch_size field; it is the only mandatory key. from_dict parses each rule object (after key validation) and throws this ValueError when max_batch_size is absent.

Source

Thrown at python/sglang/multimodal_gen/runtime/managers/dynamic_batch_admission.py:89

    model_contains: str | None = None
    resolution: str | None = None
    device_memory_gb_min: float | None = None
    device_memory_gb_max: float | None = None
    offload: bool | None = None
    max_batch_size: int = 1
    max_cost: float | None = None
    source: str = "user"

    @classmethod
    def from_dict(cls, data: dict[str, Any], *, source: str) -> BatchingRule:
        if not isinstance(data, dict):
            raise ValueError(
                f"batching config rule from {source} must be an object, "
                f"got {type(data).__name__}"
            )
        _validate_rule_keys(data, source=source)
        if "max_batch_size" not in data:
            raise ValueError("batching config rule requires max_batch_size")

        rule = cls(
            model=_optional_str(data.get("model")),
            model_contains=_optional_str(data.get("model_contains")),
            resolution=_optional_str(data.get("resolution")),
            device_memory_gb_min=_optional_float(data.get("device_memory_gb_min")),
            device_memory_gb_max=_optional_float(data.get("device_memory_gb_max")),
            offload=_optional_bool(data.get("offload")),
            max_batch_size=int(data["max_batch_size"]),
            max_cost=_optional_float(data.get("max_cost")),
            source=source,
        )
        rule.validate()
        return rule

    def validate(self) -> None:
        if self.model is not None and self.model_contains is not None:
            raise ValueError(

View on GitHub (pinned to 0132848349)

Solutions

1. Add "max_batch_size": <N> (>= 1) to every rule object in the batching config JSON
2. Validate the config with a JSON schema or a dry-run load_batching_config call before server startup
3. Check for typos in the key (unknown keys are reported separately by _validate_rule_keys)

Example fix

// before
{"model": "qwen-image", "resolution": "1024"}
// after
{"model": "qwen-image", "resolution": "1024", "max_batch_size": 8}

import json
cfg = json.load(open(path))
rules = cfg.get("rules", cfg) if isinstance(cfg, dict) else cfg
for i, r in enumerate(rules):
    if not isinstance(r, dict) or "max_batch_size" not in r:
        raise SystemExit(f"rule[{i}] missing max_batch_size")

When it happens

Trigger: Calling load_batching_config / BatchingRule.from_dict on a JSON config where a rule object omits 'max_batch_size' (e.g. {"model": "x", "resolution": "1024"}).

Common situations: Hand-written or template batching config files where the author listed only matching selectors (model/resolution) and forgot the actual batch-size limit; also configs migrated from a schema where batch size was optional or defaulted.

Related errors

• f"Unsupported patch_size type: {type(patch_size)}"
• batching config rule cannot set both model and model_contain
• batching config rule requires model or model_contains
• batching config rule max_batch_size must be >= 1
• batching config rule max_cost must be > 0

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7a1411cb38d01348. Report an issue: GitHub.