sgl-project/sglang · error · ValueError

Cosmos3 forward_dynamics produces video; use /v1/videos inst

Error message

Cosmos3 forward_dynamics produces video; use /v1/videos instead

What it means

Cosmos3's forward_dynamics mode generates video frames, not actions, so it is explicitly rejected on the /v1/actions endpoint. The builder checks the action_mode option (merged from observation and parameters) and routes forward_dynamics users to /v1/videos. This prevents a guaranteed downstream failure/inappropriate response format.

Source

Thrown at python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py:137

    if len(prompts) != batch_size:
        raise ValueError(
            "Cosmos3 batched action input requires one prompt per image, got "
            f"{len(prompts)} prompt(s) and {batch_size} image(s)"
        )
    return prompts[0] if batch_size == 1 else prompts


def build_cosmos3_action_sampling_params(
    payload: dict[str, Any],
    observation: dict[str, Any],
    server_args: ServerArgs,
    sampling_params_cls: type[Cosmos3SamplingParams],
) -> Cosmos3SamplingParams:
    parameters = dict(payload.get("parameters") or {})
    options = {**observation, **parameters}
    action_mode = str(options.get("action_mode", "policy")).strip().lower()
    if action_mode == "forward_dynamics":
        raise ValueError(
            "Cosmos3 forward_dynamics produces video; use /v1/videos instead"
        )
    if action_mode not in ("policy", "inverse_dynamics"):
        raise ValueError(
            "Cosmos3 action endpoint supports action_mode='policy' or "
            "'inverse_dynamics'"
        )

    action_horizon = options.get("action_horizon")
    num_frames = options.get("num_frames")
    if action_horizon is None and num_frames is None:
        action_horizon = 16
    if action_horizon is not None:
        action_horizon = int(action_horizon)
        if action_horizon <= 0:
            raise ValueError("action_horizon must be a positive integer")
        expected_num_frames = action_horizon + 1
        if num_frames is not None and int(num_frames) != expected_num_frames:

View on GitHub (pinned to 0132848349)

Solutions

1. Use the /v1/videos endpoint for forward_dynamics requests
2. Switch action_mode to 'policy' or 'inverse_dynamics' if you actually want actions

Example fix

# before
POST /v1/actions {"parameters": {"action_mode": "forward_dynamics"}}
# after
POST /v1/videos {"parameters": {"action_mode": "forward_dynamics"}}

if str(params.get('action_mode','policy')).strip().lower() == 'forward_dynamics':
    target = '/v1/videos'  # route there instead of /v1/actions

When it happens

Trigger: POST to the action endpoint with parameters.action_mode='forward_dynamics' or observation.action_mode='forward_dynamics' (any case/whitespace variant).

Common situations: Reusing a video-generation client config against the action endpoint; copy-pasting from Cosmos3 docs where forward_dynamics is the default video mode; exploratory calls switching action_mode values.

Related errors

• Cosmos3 action endpoint supports action_mode='policy' or 'in
• Video generation failed: {error_msg}
• Lost connection to server after {consecutive_errors} consecu
• Network error after {consecutive_errors} consecutive failure
• Video generation timed out after {max_wait_time} seconds

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7022e34a8e6d2e1d. Report an issue: GitHub.