{"record":{"id":"6c204daadfbe026c","repo":"sgl-project/sglang","slug":"cosmos3-action-requests-accept-either-an-image-or","errorCode":null,"errorMessage":"Cosmos3 action requests accept either an image or a video","messagePattern":"Cosmos3 action requests accept either an image or a video","errorType":"validation","errorClass":"ValueError","httpStatus":400,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py","lineNumber":178,"sourceCode":"        num_frames = expected_num_frames\n    else:\n        num_frames = int(num_frames)\n        if num_frames <= 1:\n            raise ValueError(\"Cosmos3 action num_frames must be greater than 1\")\n    if (num_frames - 1) % 4 != 0:\n        raise ValueError(\n            \"Cosmos3 action_horizon must be divisible by 4 so num_frames \"\n            \"is compatible with the temporal VAE\"\n        )\n\n    images = _images_from_observation(observation)\n    video_path = options.get(\"video_path\") or observation.get(\"video\")\n    if action_mode == \"policy\" and not images:\n        raise ValueError(\"Cosmos3 policy input requires an observation image\")\n    if action_mode == \"inverse_dynamics\" and video_path is None:\n        raise ValueError(\"Cosmos3 inverse_dynamics input requires an observation video\")\n    if images and video_path is not None:\n        raise ValueError(\"Cosmos3 action requests accept either an image or a video\")\n    batch_size = len(images) if images else 1\n    max_batch_size = max(1, int(getattr(server_args, \"batching_max_size\", 1)))\n    if batch_size > max_batch_size:\n        raise ValueError(\n            f\"Cosmos3 action batch size {batch_size} exceeds \"\n            f\"--batching-max-size={max_batch_size}\"\n        )\n    image_path = None if not images else images[0] if batch_size == 1 else images\n\n    domain_id = options.get(\"domain_id\")\n    domain_name = options.get(\"domain_name\")\n    raw_action_dim = options.get(\"raw_action_dim\")\n    if domain_id is None and not domain_name:\n        raise ValueError(\"Cosmos3 action requests require domain_name or domain_id\")\n    if domain_id is not None and not domain_name and raw_action_dim is None:\n        raise ValueError(\"raw_action_dim is required when only domain_id is provided\")\n\n    prompt = observation.get(\"prompt\")","sourceCodeStart":160,"sourceCodeEnd":196,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py#L160-L196","documentation":"A Cosmos3 action request must be either image-based or video-based, not both. The endpoint raises when _images_from_observation returns images AND a video_path is present, because the model input construction would be ambiguous. This catches payloads that accidentally include both media types.","triggerScenarios":"Sending an observation containing both an image and a video field in a single /v1/actions request.","commonSituations":"Reusing a generic multimodal template that always includes all fields; failed cleanup when switching a request from policy to inverse_dynamics; empty-string image fields that still parse as images.","solutions":["Remove one of the two media fields — keep image for policy, video for inverse_dynamics","If fields may linger from a template, delete keys explicitly: payload['observation'].pop('video', None)"],"exampleFix":"# before\n{\"observation\": {\"image\": \"f0.png\", \"video\": \"rollout.mp4\"}}\n# after\n{\"observation\": {\"image\": \"f0.png\"}}","handlingStrategy":"validation","validationCode":"has_img = bool(observation.get('image') or observation.get('image_url'))\nhas_vid = bool(params.get('video_path') or observation.get('video'))\nassert not (has_img and has_vid), 'send image OR video, not both'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Strip unused media keys when switching modes","Use disjoint request schemas per mode"],"tags":["cosmos3","mutually-exclusive-inputs","input-validation"],"backgroundTag":"conflicting-input-fields","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}