sgl-project/sglang · error · ValueError

batching config rule max_batch_size must be >= 1

Error message

batching config rule max_batch_size must be >= 1

What it means

max_batch_size must be a positive integer (>= 1); zero or negative values are meaningless for admission control and validate() rejects them.

Source

Thrown at python/sglang/multimodal_gen/runtime/managers/dynamic_batch_admission.py:113

            device_memory_gb_min=_optional_float(data.get("device_memory_gb_min")),
            device_memory_gb_max=_optional_float(data.get("device_memory_gb_max")),
            offload=_optional_bool(data.get("offload")),
            max_batch_size=int(data["max_batch_size"]),
            max_cost=_optional_float(data.get("max_cost")),
            source=source,
        )
        rule.validate()
        return rule

    def validate(self) -> None:
        if self.model is not None and self.model_contains is not None:
            raise ValueError(
                "batching config rule cannot set both model and model_contains"
            )
        if self.model is None and self.model_contains is None:
            raise ValueError("batching config rule requires model or model_contains")
        if self.max_batch_size < 1:
            raise ValueError("batching config rule max_batch_size must be >= 1")
        if self.max_cost is not None and self.max_cost <= 0.0:
            raise ValueError("batching config rule max_cost must be > 0")
        if (
            self.device_memory_gb_min is not None
            and self.device_memory_gb_max is not None
            and self.device_memory_gb_min > self.device_memory_gb_max
        ):
            raise ValueError(
                "batching config rule device_memory_gb_min must be <= device_memory_gb_max"
            )

    def matches(
        self,
        *,
        model_path: str,
        resolution: str | None,
        device_memory_gb: float | None,
        offload: bool,

View on GitHub (pinned to 0132848349)

Solutions

1. Set max_batch_size to 1 or higher
2. To effectively disable batching for a model, set max_batch_size: 1 rather than 0

Example fix

// before
{"model": "x", "max_batch_size": 0}
// after
{"model": "x", "max_batch_size": 1}

for i, r in enumerate(rules):
    v = r.get("max_batch_size")
    if not isinstance(v, int) or v < 1:
        raise SystemExit(f"rule[{i}]: max_batch_size must be an int >= 1")

When it happens

Trigger: A rule with "max_batch_size": 0, a negative number, or a non-integer value that coerces to < 1 (e.g. "max_batch_size": "0").

Common situations: Using 0 intending 'unlimited/disabled' (the code has no unlimited sentinel); copy from a template placeholder; off-by-one tuning experiments.

Related errors

• batching config rule max_cost must be > 0
• batching config rule device_memory_gb_min must be <= device_
• batching config rule requires max_batch_size
• batching config rule cannot set both model and model_contain
• batching config rule requires model or model_contains

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0d0b71f8f1c97d42. Report an issue: GitHub.