{"record":{"id":"c4233b9a596394e5","repo":"sgl-project/sglang","slug":"cosmos3-policy-input-requires-an-observation-image","errorCode":null,"errorMessage":"Cosmos3 policy input requires an observation image","messagePattern":"Cosmos3 policy input requires an observation image","errorType":"validation","errorClass":"ValueError","httpStatus":400,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py","lineNumber":174,"sourceCode":"            raise ValueError(\n                \"Cosmos3 requires num_frames == action_horizon + 1, got \"\n                f\"num_frames={num_frames}, action_horizon={action_horizon}\"\n            )\n        num_frames = expected_num_frames\n    else:\n        num_frames = int(num_frames)\n        if num_frames <= 1:\n            raise ValueError(\"Cosmos3 action num_frames must be greater than 1\")\n    if (num_frames - 1) % 4 != 0:\n        raise ValueError(\n            \"Cosmos3 action_horizon must be divisible by 4 so num_frames \"\n            \"is compatible with the temporal VAE\"\n        )\n\n    images = _images_from_observation(observation)\n    video_path = options.get(\"video_path\") or observation.get(\"video\")\n    if action_mode == \"policy\" and not images:\n        raise ValueError(\"Cosmos3 policy input requires an observation image\")\n    if action_mode == \"inverse_dynamics\" and video_path is None:\n        raise ValueError(\"Cosmos3 inverse_dynamics input requires an observation video\")\n    if images and video_path is not None:\n        raise ValueError(\"Cosmos3 action requests accept either an image or a video\")\n    batch_size = len(images) if images else 1\n    max_batch_size = max(1, int(getattr(server_args, \"batching_max_size\", 1)))\n    if batch_size > max_batch_size:\n        raise ValueError(\n            f\"Cosmos3 action batch size {batch_size} exceeds \"\n            f\"--batching-max-size={max_batch_size}\"\n        )\n    image_path = None if not images else images[0] if batch_size == 1 else images\n\n    domain_id = options.get(\"domain_id\")\n    domain_name = options.get(\"domain_name\")\n    raw_action_dim = options.get(\"raw_action_dim\")\n    if domain_id is None and not domain_name:\n        raise ValueError(\"Cosmos3 action requests require domain_name or domain_id\")","sourceCodeStart":156,"sourceCodeEnd":192,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py#L156-L192","documentation":"Cosmos3's policy mode predicts actions from an observation image; without one the request cannot be processed. The endpoint extracts images via _images_from_observation and raises if the list is empty when action_mode='policy'. Images can be supplied as URLs/paths or embedded image payloads in the observation.","triggerScenarios":"POST /v1/actions with action_mode='policy' (or default) but no image field, or an image field _images_from_observation doesn't recognize (empty string, empty list).","commonSituations":"Client sends only a video for a policy query; image key typo (e.g. 'img' instead of the expected image/image_url key); empty b64 payload; mixing up policy and inverse_dynamics request shapes.","solutions":["Add an observation image (image_path/image_url or b64 image payload) to the request","Verify the image field name matches what _images_from_observation parses","If your observation is a video, use action_mode='inverse_dynamics' instead"],"exampleFix":"# before\n{\"observation\": {\"prompt\": \"pick cube\"}}\n# after\n{\"observation\": {\"prompt\": \"pick cube\", \"image\": \"</path/or/url>\"}}","handlingStrategy":"validation","validationCode":"assert observation.get('image') or observation.get('image_url'), 'policy mode needs an observation image'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Build per-mode request templates: policy=image, inverse_dynamics=video","Assert required media fields before sending"],"tags":["cosmos3","policy-mode","missing-image","input-validation"],"backgroundTag":"missing-required-input","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}