sgl-project/sglang · error · ValueError

batching config rule max_cost must be > 0

Error message

batching config rule max_cost must be > 0

What it means

The optional max_cost cap on a rule, when present, must be strictly positive. validate() raises when max_cost is set to 0 or a negative number.

Source

Thrown at python/sglang/multimodal_gen/runtime/managers/dynamic_batch_admission.py:115

            offload=_optional_bool(data.get("offload")),
            max_batch_size=int(data["max_batch_size"]),
            max_cost=_optional_float(data.get("max_cost")),
            source=source,
        )
        rule.validate()
        return rule

    def validate(self) -> None:
        if self.model is not None and self.model_contains is not None:
            raise ValueError(
                "batching config rule cannot set both model and model_contains"
            )
        if self.model is None and self.model_contains is None:
            raise ValueError("batching config rule requires model or model_contains")
        if self.max_batch_size < 1:
            raise ValueError("batching config rule max_batch_size must be >= 1")
        if self.max_cost is not None and self.max_cost <= 0.0:
            raise ValueError("batching config rule max_cost must be > 0")
        if (
            self.device_memory_gb_min is not None
            and self.device_memory_gb_max is not None
            and self.device_memory_gb_min > self.device_memory_gb_max
        ):
            raise ValueError(
                "batching config rule device_memory_gb_min must be <= device_memory_gb_max"
            )

    def matches(
        self,
        *,
        model_path: str,
        resolution: str | None,
        device_memory_gb: float | None,
        offload: bool,
    ) -> bool:
        if self.model is not None and self.model != model_path:

View on GitHub (pinned to 0132848349)

Solutions

1. Remove the "max_cost" key entirely if no cap is desired
2. Otherwise set it to a positive float

Example fix

// before
{"model": "x", "max_batch_size": 4, "max_cost": 0}
// after
{"model": "x", "max_batch_size": 4}

for i, r in enumerate(rules):
    c = r.get("max_cost")
    if c is not None and (not isinstance(c, (int, float)) or c <= 0):
        raise SystemExit(f"rule[{i}]: max_cost must be > 0 or omitted")

When it happens

Trigger: A rule containing "max_cost": 0 or "max_cost": -1.0 (max_cost omitted or None is fine).

Common situations: Author uses max_cost: 0 to mean 'no cost cap' instead of omitting the key; unit mistakes producing tiny/negative values.

Related errors

• batching config rule max_batch_size must be >= 1
• batching config rule device_memory_gb_min must be <= device_
• batching config rule requires max_batch_size
• batching config rule cannot set both model and model_contain
• batching config rule requires model or model_contains

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/dc870fd382ed28df. Report an issue: GitHub.