sgl-project/sglang · error · ValueError

batching config rule device_memory_gb_min must be <= device_

Error message

batching config rule device_memory_gb_min must be <= device_memory_gb_max

What it means

When a rule specifies both device_memory_gb_min and device_memory_gb_max device-memory bounds, the min must not exceed the max. validate() enforces the ordering when both are present.

Source

Thrown at python/sglang/multimodal_gen/runtime/managers/dynamic_batch_admission.py:121

        return rule

    def validate(self) -> None:
        if self.model is not None and self.model_contains is not None:
            raise ValueError(
                "batching config rule cannot set both model and model_contains"
            )
        if self.model is None and self.model_contains is None:
            raise ValueError("batching config rule requires model or model_contains")
        if self.max_batch_size < 1:
            raise ValueError("batching config rule max_batch_size must be >= 1")
        if self.max_cost is not None and self.max_cost <= 0.0:
            raise ValueError("batching config rule max_cost must be > 0")
        if (
            self.device_memory_gb_min is not None
            and self.device_memory_gb_max is not None
            and self.device_memory_gb_min > self.device_memory_gb_max
        ):
            raise ValueError(
                "batching config rule device_memory_gb_min must be <= device_memory_gb_max"
            )

    def matches(
        self,
        *,
        model_path: str,
        resolution: str | None,
        device_memory_gb: float | None,
        offload: bool,
    ) -> bool:
        if self.model is not None and self.model != model_path:
            return False
        if self.model_contains is not None and self.model_contains not in model_path:
            return False
        if self.resolution not in (None, "*") and self.resolution != resolution:
            return False
        if self.offload is not None and self.offload != offload:

View on GitHub (pinned to 0132848349)

Solutions

1. Swap or correct the two values so min <= max
2. If you only need a floor or ceiling, specify just one bound

Example fix

// before
{"model": "x", "max_batch_size": 4, "device_memory_gb_min": 80, "device_memory_gb_max": 40}
// after
{"model": "x", "max_batch_size": 4, "device_memory_gb_min": 40, "device_memory_gb_max": 80}

for i, r in enumerate(rules):
    lo, hi = r.get("device_memory_gb_min"), r.get("device_memory_gb_max")
    if lo is not None and hi is not None and lo > hi:
        raise SystemExit(f"rule[{i}]: device_memory_gb_min > max")

When it happens

Trigger: A rule with "device_memory_gb_min": 80 and "device_memory_gb_max": 40 (both non-None and min > max).

Common situations: Swapped min/max keys while hand-editing; stale bounds after hardware changes (e.g. old 40GB upper bound kept with new 80GB floor).

Related errors

• batching config rule max_batch_size must be >= 1
• batching config rule max_cost must be > 0
• batching config rule requires max_batch_size
• batching config rule cannot set both model and model_contain
• batching config rule requires model or model_contains

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3e4d11c7f8cc4dc9. Report an issue: GitHub.