sgl-project/sglang · error · ValueError

Cosmos3 action prompt list must contain only strings

Error message

Cosmos3 action prompt list must contain only strings

What it means

The Cosmos3 action endpoint validates that when the prompt is provided as a list/tuple, every element must be a string. It is thrown by the _action_prompt helper in build_cosmos3_action_sampling_params before any sampling params are built. Mixed content (e.g. numbers, None, nested lists) in the prompt list is rejected because the model prompt construction requires plain strings.

Source

Thrown at python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py:115

            continue
        if item.dtype != np.uint8:
            raise ValueError("Cosmos3 observation image arrays must use uint8 dtype")
        if item.ndim not in (2, 3):
            raise ValueError(
                "Cosmos3 observation image arrays must have shape [H, W] "
                f"or [H, W, C], got {tuple(item.shape)}"
            )
        normalized_images.append(Image.fromarray(item))
    return normalized_images


def _action_prompt(prompt: Any, batch_size: int) -> str | list[str]:
    if isinstance(prompt, str):
        return prompt if batch_size == 1 else [prompt] * batch_size
    if not isinstance(prompt, (list, tuple)) or not prompt:
        raise ValueError("Cosmos3 action prompt must be a string or non-empty list")
    if not all(isinstance(item, str) for item in prompt):
        raise ValueError("Cosmos3 action prompt list must contain only strings")
    prompts = list(prompt)
    if len(prompts) == 1 and batch_size > 1:
        prompts *= batch_size
    if len(prompts) != batch_size:
        raise ValueError(
            "Cosmos3 batched action input requires one prompt per image, got "
            f"{len(prompts)} prompt(s) and {batch_size} image(s)"
        )
    return prompts[0] if batch_size == 1 else prompts


def build_cosmos3_action_sampling_params(
    payload: dict[str, Any],
    observation: dict[str, Any],
    server_args: ServerArgs,
    sampling_params_cls: type[Cosmos3SamplingParams],
) -> Cosmos3SamplingParams:
    parameters = dict(payload.get("parameters") or {})

View on GitHub (pinned to 0132848349)

Solutions

1. Make every element of the prompt list a string, e.g. [str(p) for p in prompt]
2. If using OpenAI-style content blocks, extract the text fields first: [b['text'] for b in content if b.get('type')=='text']
3. Pass a single string when all images share the same prompt — it is broadcast to batch_size automatically

Example fix

# before
prompt = ["pick up the cube", 42]
# after
prompt = ["pick up the cube", "42"]

def is_valid_prompt_list(p):
    return isinstance(p, (list, tuple)) and len(p) > 0 and all(isinstance(x, str) for x in p)

def is_string_list(p) -> bool:
    return isinstance(p, list) and all(isinstance(x, str) for x in p)

When it happens

Trigger: POSTing to the action endpoint with observation.prompt = ["do X", 5] or ["a", null]; any list prompt containing a non-str element (int, dict, None).

Common situations: Serializing numeric task ids or structured prompt parts into the prompt list; passing a JSON-decoded list that mixed types; reusing an OpenAI-style content array (with {type:'text'} dicts) as the prompt list.

Related errors

• Cosmos3 inverse_dynamics prompt must be a string
• Cosmos3 batched action input requires one prompt per image,
• num_token_non_padded must be a torch.Tensor
• actions must be a list[list[str]]
• rollout_noise_level must be a number, got {noise!r}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/aef9e859071f7107. Report an issue: GitHub.