sgl-project/sglang · error · ValueError

Cosmos3 observation image arrays must use uint8 dtype

Error message

Cosmos3 observation image arrays must use uint8 dtype

What it means

When a Cosmos3 observation image is a numpy array, it must have dtype uint8 (it is converted with PIL Image.fromarray). Any other dtype (float32, uint16, etc.) raises this ValueError before the request is processed.

Solutions

1. Convert to uint8 before sending: arr = (arr * 255).clip(0,255).astype(np.uint8) for float input, or arr.astype(np.uint8) for integer input
2. Keep raw camera output (typically already uint8 RGB) instead of pre-normalized arrays

Example fix

# before
image = (frame / 255.0).astype(np.float32)  # float32 -> error

# after
image = frame.astype(np.uint8)               # keep uint8

import numpy as np
def ensure_uint8(arr: np.ndarray) -> np.ndarray:
    if arr.dtype != np.uint8:
        if np.issubdtype(arr.dtype, np.floating):
            arr = (np.clip(arr, 0, 1) * 255).astype(np.uint8)
        else:
            arr = arr.astype(np.uint8)
    return arr

def is_uint8_image(x) -> bool:
    return not isinstance(x, np.ndarray) or x.dtype == np.uint8

When it happens

Trigger: Passing normalized float images (0.0-1.0 float32, common after preprocessing) or 16-bit depth images as np.ndarray in the observation.

Common situations: Images coming from a model-preprocessing pipeline that rescales to [0,1] float; depth cameras producing uint16; downstream consumers expecting float tensors.

Related errors

• Cosmos3 observation image arrays must have shape [H, W] or…
• Cosmos3 action input accepts one image field; use a list or…
• Cosmos3 I2V image list is empty
• action_horizon must be a positive integer
• action_mode='forward_dynamics' requires an 'action' array…

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d74d38eb15870236. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py:99

                "Cosmos3 action input accepts one image field; use a list or "
                "a [B, H, W, C] array in that field for batched observations"
            )
        image = next(iter(images.values()))

    if isinstance(image, (list, tuple)):
        images = list(image)
    elif isinstance(image, np.ndarray) and image.ndim == 4:
        images = list(image)
    else:
        images = [image]

    normalized_images: list[Any] = []
    for item in images:
        if not isinstance(item, np.ndarray):
            normalized_images.append(item)
            continue
        if item.dtype != np.uint8:
            raise ValueError("Cosmos3 observation image arrays must use uint8 dtype")
        if item.ndim not in (2, 3):
            raise ValueError(
                "Cosmos3 observation image arrays must have shape [H, W] "
                f"or [H, W, C], got {tuple(item.shape)}"
            )
        normalized_images.append(Image.fromarray(item))
    return normalized_images


def _action_prompt(prompt: Any, batch_size: int) -> str | list[str]:
    if isinstance(prompt, str):
        return prompt if batch_size == 1 else [prompt] * batch_size
    if not isinstance(prompt, (list, tuple)) or not prompt:
        raise ValueError("Cosmos3 action prompt must be a string or non-empty list")
    if not all(isinstance(item, str) for item in prompt):
        raise ValueError("Cosmos3 action prompt list must contain only strings")
    prompts = list(prompt)
    if len(prompts) == 1 and batch_size > 1:

View on GitHub (pinned to 0132848349)