sgl-project/sglang · error · ValueError

Cosmos3 action input accepts one image field; use a list or

Error message

Cosmos3 action input accepts one image field; use a list or a [B, H, W, C] array in that field for batched observations

What it means

Cosmos3 action input expects exactly one image field: either observation['image'] or a single key inside observation['images']. If 'images' is present but is not a dict with exactly one entry, this ValueError is raised. Batched observations should be a list or [B,H,W,C] array inside that single field.

Source

Thrown at python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py:80

            "batched_action_modes": ["policy"],
            "multiple_candidates": False,
        },
    }


def _images_from_observation(observation: dict[str, Any]) -> list[Any]:
    image = None
    for name in ("image", "image_path", "input_reference"):
        if name in observation:
            image = observation[name]
            break

    if image is None:
        images = observation.get("images")
        if images is None or (isinstance(images, dict) and not images):
            return []
        if not isinstance(images, dict) or len(images) != 1:
            raise ValueError(
                "Cosmos3 action input accepts one image field; use a list or "
                "a [B, H, W, C] array in that field for batched observations"
            )
        image = next(iter(images.values()))

    if isinstance(image, (list, tuple)):
        images = list(image)
    elif isinstance(image, np.ndarray) and image.ndim == 4:
        images = list(image)
    else:
        images = [image]

    normalized_images: list[Any] = []
    for item in images:
        if not isinstance(item, np.ndarray):
            normalized_images.append(item)
            continue
        if item.dtype != np.uint8:

View on GitHub (pinned to 0132848349)

Solutions

1. Collapse to a single image field: keep only one camera (e.g. observation={'image': front_img})
2. For batches, pass a list of images or a [B,H,W,C] uint8 array within that one field
3. If multiple views are required, preprocess/stack them externally before sending

Example fix

# before
observation = {"images": {"front": f, "wrist": w}}

# after
observation = {"image": f}  # or [f, w] stacked as [B,H,W,C] if batch semantics fit

def normalize_observation(obs):
    if "image" in obs and obs["image"] is not None:
        return obs
    imgs = obs.get("images")
    assert isinstance(imgs, dict) and len(imgs) == 1, "need exactly one image field"
    return {"image": next(iter(imgs.values()))}

def has_single_image_field(obs: dict) -> bool:
    if obs.get("image") is not None:
        return True
    imgs = obs.get("images")
    return isinstance(imgs, dict) and len(imgs) == 1

When it happens

Trigger: Passing observation={'images': {'front': ..., 'wrist': ...}} (two camera views), or images as a bare list/dict with 0 or 2+ keys.

Common situations: Robotics VLA pipelines with multiple camera streams (front + wrist) that the model doesn't support; refactoring an observation schema that previously used different key names.

Related errors

• Cosmos3 observation image arrays must use uint8 dtype
• Cosmos3 observation image arrays must have shape [H, W] or [
• Cosmos3 action prompt must be a string or non-empty list
• Cosmos3 I2V image list is empty
• Cosmos3 batched prompts must tokenize to the same length bec

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/8e836d78e606b8ea. Report an issue: GitHub.