{"record":{"id":"8e836d78e606b8ea","repo":"sgl-project/sglang","slug":"cosmos3-action-input-accepts-one-image-field-use","errorCode":null,"errorMessage":"Cosmos3 action input accepts one image field; use a list or a [B, H, W, C] array in that field for batched observations","messagePattern":"Cosmos3 action input accepts one image field; use a list or a \\[B, H, W, C\\] array in that field for batched observations","errorType":"validation","errorClass":"ValueError","httpStatus":400,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py","lineNumber":80,"sourceCode":"            \"batched_action_modes\": [\"policy\"],\n            \"multiple_candidates\": False,\n        },\n    }\n\n\ndef _images_from_observation(observation: dict[str, Any]) -> list[Any]:\n    image = None\n    for name in (\"image\", \"image_path\", \"input_reference\"):\n        if name in observation:\n            image = observation[name]\n            break\n\n    if image is None:\n        images = observation.get(\"images\")\n        if images is None or (isinstance(images, dict) and not images):\n            return []\n        if not isinstance(images, dict) or len(images) != 1:\n            raise ValueError(\n                \"Cosmos3 action input accepts one image field; use a list or \"\n                \"a [B, H, W, C] array in that field for batched observations\"\n            )\n        image = next(iter(images.values()))\n\n    if isinstance(image, (list, tuple)):\n        images = list(image)\n    elif isinstance(image, np.ndarray) and image.ndim == 4:\n        images = list(image)\n    else:\n        images = [image]\n\n    normalized_images: list[Any] = []\n    for item in images:\n        if not isinstance(item, np.ndarray):\n            normalized_images.append(item)\n            continue\n        if item.dtype != np.uint8:","sourceCodeStart":62,"sourceCodeEnd":98,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py#L62-L98","documentation":"Cosmos3 action input expects exactly one image field: either observation['image'] or a single key inside observation['images']. If 'images' is present but is not a dict with exactly one entry, this ValueError is raised. Batched observations should be a list or [B,H,W,C] array inside that single field.","triggerScenarios":"Passing observation={'images': {'front': ..., 'wrist': ...}} (two camera views), or images as a bare list/dict with 0 or 2+ keys.","commonSituations":"Robotics VLA pipelines with multiple camera streams (front + wrist) that the model doesn't support; refactoring an observation schema that previously used different key names.","solutions":["Collapse to a single image field: keep only one camera (e.g. observation={'image': front_img})","For batches, pass a list of images or a [B,H,W,C] uint8 array within that one field","If multiple views are required, preprocess/stack them externally before sending"],"exampleFix":"# before\nobservation = {\"images\": {\"front\": f, \"wrist\": w}}\n\n# after\nobservation = {\"image\": f}  # or [f, w] stacked as [B,H,W,C] if batch semantics fit","handlingStrategy":"type-guard","validationCode":"def normalize_observation(obs):\n    if \"image\" in obs and obs[\"image\"] is not None:\n        return obs\n    imgs = obs.get(\"images\")\n    assert isinstance(imgs, dict) and len(imgs) == 1, \"need exactly one image field\"\n    return {\"image\": next(iter(imgs.values()))}","typeGuard":"def has_single_image_field(obs: dict) -> bool:\n    if obs.get(\"image\") is not None:\n        return True\n    imgs = obs.get(\"images\")\n    return isinstance(imgs, dict) and len(imgs) == 1","tryCatchPattern":null,"preventionTips":["Standardize on a single 'image' key in observation producers","For batches use one list/[B,H,W,C] array, never multiple named fields","Document the single-camera constraint for downstream integrators"],"tags":["cosmos3","observation","image-input","validation"],"backgroundTag":"invalid-image-input","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}