sgl-project/sglang · error · ValueError

batching config rule requires model or model_contains

Error message

batching config rule requires model or model_contains

What it means

Every BatchingRule needs at least one component selector: either model (exact match) or model_contains (substring match). Without one the rule could not be attributed to any model, so validate() rejects it.

Source

Thrown at python/sglang/multimodal_gen/runtime/managers/dynamic_batch_admission.py:111

            model_contains=_optional_str(data.get("model_contains")),
            resolution=_optional_str(data.get("resolution")),
            device_memory_gb_min=_optional_float(data.get("device_memory_gb_min")),
            device_memory_gb_max=_optional_float(data.get("device_memory_gb_max")),
            offload=_optional_bool(data.get("offload")),
            max_batch_size=int(data["max_batch_size"]),
            max_cost=_optional_float(data.get("max_cost")),
            source=source,
        )
        rule.validate()
        return rule

    def validate(self) -> None:
        if self.model is not None and self.model_contains is not None:
            raise ValueError(
                "batching config rule cannot set both model and model_contains"
            )
        if self.model is None and self.model_contains is None:
            raise ValueError("batching config rule requires model or model_contains")
        if self.max_batch_size < 1:
            raise ValueError("batching config rule max_batch_size must be >= 1")
        if self.max_cost is not None and self.max_cost <= 0.0:
            raise ValueError("batching config rule max_cost must be > 0")
        if (
            self.device_memory_gb_min is not None
            and self.device_memory_gb_max is not None
            and self.device_memory_gb_min > self.device_memory_gb_max
        ):
            raise ValueError(
                "batching config rule device_memory_gb_min must be <= device_memory_gb_max"
            )

    def matches(
        self,
        *,
        model_path: str,
        resolution: str | None,

View on GitHub (pinned to 0132848349)

Solutions

1. Add "model": "<exact name>" or "model_contains": "<substring>" to the rule
2. Use model_contains with a broad substring only if a catch-all was intended

Example fix

// before
{"resolution": "512", "max_batch_size": 2}
// after
{"model_contains": "", "resolution": "512", "max_batch_size": 2}

for i, r in enumerate(rules):
    if "model" not in r and "model_contains" not in r:
        raise SystemExit(f"rule[{i}]: needs model or model_contains")

When it happens

Trigger: A rule dict that has max_batch_size but neither "model" nor "model_contains" keys (both parse to None), e.g. {"resolution": "512", "max_batch_size": 2}.

Common situations: Author intended a resolution-only or device-memory-only rule and assumed a default/catch-all selector exists; or trimmed a rule down during editing and removed the selector.

Related errors

• batching config rule requires max_batch_size
• batching config rule cannot set both model and model_contain
• batching config rule max_batch_size must be >= 1
• batching config rule max_cost must be > 0
• batching config rule device_memory_gb_min must be <= device_

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f65c7a9d443a70cc. Report an issue: GitHub.