{"record":{"id":"88727ce39034a5de","repo":"sgl-project/sglang","slug":"cosmos3-action-batch-size-batch-size-exceeds-b","errorCode":null,"errorMessage":"Cosmos3 action batch size {batch_size} exceeds --batching-max-size={max_batch_size}","messagePattern":"Cosmos3 action batch size (.+?) exceeds --batching-max-size=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":400,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py","lineNumber":182,"sourceCode":"            raise ValueError(\"Cosmos3 action num_frames must be greater than 1\")\n    if (num_frames - 1) % 4 != 0:\n        raise ValueError(\n            \"Cosmos3 action_horizon must be divisible by 4 so num_frames \"\n            \"is compatible with the temporal VAE\"\n        )\n\n    images = _images_from_observation(observation)\n    video_path = options.get(\"video_path\") or observation.get(\"video\")\n    if action_mode == \"policy\" and not images:\n        raise ValueError(\"Cosmos3 policy input requires an observation image\")\n    if action_mode == \"inverse_dynamics\" and video_path is None:\n        raise ValueError(\"Cosmos3 inverse_dynamics input requires an observation video\")\n    if images and video_path is not None:\n        raise ValueError(\"Cosmos3 action requests accept either an image or a video\")\n    batch_size = len(images) if images else 1\n    max_batch_size = max(1, int(getattr(server_args, \"batching_max_size\", 1)))\n    if batch_size > max_batch_size:\n        raise ValueError(\n            f\"Cosmos3 action batch size {batch_size} exceeds \"\n            f\"--batching-max-size={max_batch_size}\"\n        )\n    image_path = None if not images else images[0] if batch_size == 1 else images\n\n    domain_id = options.get(\"domain_id\")\n    domain_name = options.get(\"domain_name\")\n    raw_action_dim = options.get(\"raw_action_dim\")\n    if domain_id is None and not domain_name:\n        raise ValueError(\"Cosmos3 action requests require domain_name or domain_id\")\n    if domain_id is not None and not domain_name and raw_action_dim is None:\n        raise ValueError(\"raw_action_dim is required when only domain_id is provided\")\n\n    prompt = observation.get(\"prompt\")\n    if prompt is None:\n        prompt = observation.get(\"task\", \"\")\n    if action_mode != \"policy\" and not isinstance(prompt, str):\n        raise ValueError(\"Cosmos3 inverse_dynamics prompt must be a string\")","sourceCodeStart":164,"sourceCodeEnd":200,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py#L164-L200","documentation":"The number of observation images in a batched Cosmos3 action request exceeds the server's --batching-max-size limit. batch_size = len(images) is compared against max(1, server_args.batching_max_size); larger batches are rejected before scheduling to protect memory. Default limit is derived from server launch args.","triggerScenarios":"Sending N images where N > the server's batching_max_size (e.g. 8 images against --batching-max-size 4).","commonSituations":"Scaling up eval batch size without restarting the server with a higher limit; default batching_max_size too small for the dataset loader's chunk size.","solutions":["Restart the server with a larger limit: --batching-max-size <N>","Or chunk client-side: send images in batches of at most batching_max_size"],"exampleFix":"# before\npython -m sglang.launch_server ... --batching-max-size 4\n# after\npython -m sglang.launch_server ... --batching-max-size 16","handlingStrategy":"validation","validationCode":"max_b = server_batching_max_size  # from server config\nfor i in range(0, len(images), max_b):\n    send_batch(images[i:i+max_b], prompts[i:i+max_b])","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Chunk requests client-side to the server's batching_max_size","Document the server's --batching-max-size in client configs"],"tags":["cosmos3","batching","capacity-limit","server-args"],"backgroundTag":"batch-size-limit-exceeded","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}