sgl-project/sglang · error · ValueError

Cosmos3 action batch size {batch_size} exceeds --batching-ma

Error message

Cosmos3 action batch size {batch_size} exceeds --batching-max-size={max_batch_size}

What it means

The number of observation images in a batched Cosmos3 action request exceeds the server's --batching-max-size limit. batch_size = len(images) is compared against max(1, server_args.batching_max_size); larger batches are rejected before scheduling to protect memory. Default limit is derived from server launch args.

Source

Thrown at python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py:182

            raise ValueError("Cosmos3 action num_frames must be greater than 1")
    if (num_frames - 1) % 4 != 0:
        raise ValueError(
            "Cosmos3 action_horizon must be divisible by 4 so num_frames "
            "is compatible with the temporal VAE"
        )

    images = _images_from_observation(observation)
    video_path = options.get("video_path") or observation.get("video")
    if action_mode == "policy" and not images:
        raise ValueError("Cosmos3 policy input requires an observation image")
    if action_mode == "inverse_dynamics" and video_path is None:
        raise ValueError("Cosmos3 inverse_dynamics input requires an observation video")
    if images and video_path is not None:
        raise ValueError("Cosmos3 action requests accept either an image or a video")
    batch_size = len(images) if images else 1
    max_batch_size = max(1, int(getattr(server_args, "batching_max_size", 1)))
    if batch_size > max_batch_size:
        raise ValueError(
            f"Cosmos3 action batch size {batch_size} exceeds "
            f"--batching-max-size={max_batch_size}"
        )
    image_path = None if not images else images[0] if batch_size == 1 else images

    domain_id = options.get("domain_id")
    domain_name = options.get("domain_name")
    raw_action_dim = options.get("raw_action_dim")
    if domain_id is None and not domain_name:
        raise ValueError("Cosmos3 action requests require domain_name or domain_id")
    if domain_id is not None and not domain_name and raw_action_dim is None:
        raise ValueError("raw_action_dim is required when only domain_id is provided")

    prompt = observation.get("prompt")
    if prompt is None:
        prompt = observation.get("task", "")
    if action_mode != "policy" and not isinstance(prompt, str):
        raise ValueError("Cosmos3 inverse_dynamics prompt must be a string")

View on GitHub (pinned to 0132848349)

Solutions

1. Restart the server with a larger limit: --batching-max-size <N>
2. Or chunk client-side: send images in batches of at most batching_max_size

Example fix

# before
python -m sglang.launch_server ... --batching-max-size 4
# after
python -m sglang.launch_server ... --batching-max-size 16

max_b = server_batching_max_size  # from server config
for i in range(0, len(images), max_b):
    send_batch(images[i:i+max_b], prompts[i:i+max_b])

When it happens

Trigger: Sending N images where N > the server's batching_max_size (e.g. 8 images against --batching-max-size 4).

Common situations: Scaling up eval batch size without restarting the server with a higher limit; default batching_max_size too small for the dataset loader's chunk size.

Related errors

• Cosmos3 batched action input requires one prompt per image,
• Cosmos3 batched prompts must tokenize to the same length bec
• action_mode={mode!r} requires --raw-action-dim.
• topk kernels only support k <= 32: {k=}
• Krea-2 sequence parallelism does not support ragged/padded m

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/88727ce39034a5de. Report an issue: GitHub.