{"record":{"id":"d74d38eb15870236","repo":"sgl-project/sglang","slug":"cosmos3-observation-image-arrays-must-use-uint8-dt","errorCode":null,"errorMessage":"Cosmos3 observation image arrays must use uint8 dtype","messagePattern":"Cosmos3 observation image arrays must use uint8 dtype","errorType":"validation","errorClass":"ValueError","httpStatus":400,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py","lineNumber":99,"sourceCode":"                \"Cosmos3 action input accepts one image field; use a list or \"\n                \"a [B, H, W, C] array in that field for batched observations\"\n            )\n        image = next(iter(images.values()))\n\n    if isinstance(image, (list, tuple)):\n        images = list(image)\n    elif isinstance(image, np.ndarray) and image.ndim == 4:\n        images = list(image)\n    else:\n        images = [image]\n\n    normalized_images: list[Any] = []\n    for item in images:\n        if not isinstance(item, np.ndarray):\n            normalized_images.append(item)\n            continue\n        if item.dtype != np.uint8:\n            raise ValueError(\"Cosmos3 observation image arrays must use uint8 dtype\")\n        if item.ndim not in (2, 3):\n            raise ValueError(\n                \"Cosmos3 observation image arrays must have shape [H, W] \"\n                f\"or [H, W, C], got {tuple(item.shape)}\"\n            )\n        normalized_images.append(Image.fromarray(item))\n    return normalized_images\n\n\ndef _action_prompt(prompt: Any, batch_size: int) -> str | list[str]:\n    if isinstance(prompt, str):\n        return prompt if batch_size == 1 else [prompt] * batch_size\n    if not isinstance(prompt, (list, tuple)) or not prompt:\n        raise ValueError(\"Cosmos3 action prompt must be a string or non-empty list\")\n    if not all(isinstance(item, str) for item in prompt):\n        raise ValueError(\"Cosmos3 action prompt list must contain only strings\")\n    prompts = list(prompt)\n    if len(prompts) == 1 and batch_size > 1:","sourceCodeStart":81,"sourceCodeEnd":117,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py#L81-L117","documentation":"When a Cosmos3 observation image is a numpy array, it must have dtype uint8 (it is converted with PIL Image.fromarray). Any other dtype (float32, uint16, etc.) raises this ValueError before the request is processed.","triggerScenarios":"Passing normalized float images (0.0-1.0 float32, common after preprocessing) or 16-bit depth images as np.ndarray in the observation.","commonSituations":"Images coming from a model-preprocessing pipeline that rescales to [0,1] float; depth cameras producing uint16; downstream consumers expecting float tensors.","solutions":["Convert to uint8 before sending: arr = (arr * 255).clip(0,255).astype(np.uint8) for float input, or arr.astype(np.uint8) for integer input","Keep raw camera output (typically already uint8 RGB) instead of pre-normalized arrays"],"exampleFix":"# before\nimage = (frame / 255.0).astype(np.float32)  # float32 -> error\n\n# after\nimage = frame.astype(np.uint8)               # keep uint8","handlingStrategy":"validation","validationCode":"import numpy as np\ndef ensure_uint8(arr: np.ndarray) -> np.ndarray:\n    if arr.dtype != np.uint8:\n        if np.issubdtype(arr.dtype, np.floating):\n            arr = (np.clip(arr, 0, 1) * 255).astype(np.uint8)\n        else:\n            arr = arr.astype(np.uint8)\n    return arr","typeGuard":"def is_uint8_image(x) -> bool:\n    return not isinstance(x, np.ndarray) or x.dtype == np.uint8","tryCatchPattern":null,"preventionTips":["Skip float normalization before sending; keep raw camera uint8 frames","Centralize a to_uint8 converter in the observation pipeline","Assert dtype at the edge of your robotics/VLA data loader"],"tags":["cosmos3","numpy","dtype","image-input"],"backgroundTag":"image-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}