sgl-project/sglang · error · ValueError

batching config rule from {source} must be an object, got {t

Error message

batching config rule from {source} must be an object, got {type(data).__name__}

What it means

A batching configuration rule is expected to be a JSON object (mapping of option names to values), but the parsed entry is some other JSON type (list, string, number, null, boolean). BatchingRule.from_dict validates shape before reading fields like max_batch_size.

Source

Thrown at python/sglang/multimodal_gen/runtime/managers/dynamic_batch_admission.py:83

@dataclass(frozen=True)
class BatchingRule:
    """One user-provided batching admission rule loaded from batching config."""

    model: str | None = None
    model_contains: str | None = None
    resolution: str | None = None
    device_memory_gb_min: float | None = None
    device_memory_gb_max: float | None = None
    offload: bool | None = None
    max_batch_size: int = 1
    max_cost: float | None = None
    source: str = "user"

    @classmethod
    def from_dict(cls, data: dict[str, Any], *, source: str) -> BatchingRule:
        if not isinstance(data, dict):
            raise ValueError(
                f"batching config rule from {source} must be an object, "
                f"got {type(data).__name__}"
            )
        _validate_rule_keys(data, source=source)
        if "max_batch_size" not in data:
            raise ValueError("batching config rule requires max_batch_size")

        rule = cls(
            model=_optional_str(data.get("model")),
            model_contains=_optional_str(data.get("model_contains")),
            resolution=_optional_str(data.get("resolution")),
            device_memory_gb_min=_optional_float(data.get("device_memory_gb_min")),
            device_memory_gb_max=_optional_float(data.get("device_memory_gb_max")),
            offload=_optional_bool(data.get("offload")),
            max_batch_size=int(data["max_batch_size"]),
            max_cost=_optional_float(data.get("max_cost")),
            source=source,
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Fix the config so each rule is an object with keys like max_batch_size, e.g. {"max_batch_size": 8}
2. Check the error's source field to find which file/section is malformed
3. Validate the config with a JSON schema or by loading it in a dry-run before starting the server

Example fix

# before
batching_rules:
  - 8
# after
batching_rules:
  - max_batch_size: 8

def rules_well_formed(rules) -> bool:
    return all(isinstance(r, dict) for r in rules)

from typing import Any

def is_batching_rule_dict(data: Any) -> bool:
    """Narrow a parsed JSON node to a BatchingRule-shaped dict."""
    return isinstance(data, dict) and "max_batch_size" in data

try:
    BatchingRule.from_dict(rule, source=path)
except ValueError as e:
    logger.error("invalid batching rule in %s: %s", path, e)
    raise SystemExit(2)

When it happens

Trigger: load_batching_config parses a batching config file where a rule entry is not a dict — e.g. a list of values, a bare string, or null — and passes it to BatchingRule.from_dict(data, source=...).

Common situations: Hand-edited YAML/JSON batching config where a rule is written as a list or scalar; schema change from an older array-based format; missing entry rendered as null.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• batching config must be a {'schema_version': 1, 'rules': [..
• batching config rule requires max_batch_size
• batching config rule cannot set both model and model_contain
• batching config rule requires model or model_contains
• batching config rule max_batch_size must be >= 1

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9455993ee1762342. Report an issue: GitHub.