{"record":{"id":"66a0c50be9a5509f","repo":"sgl-project/sglang","slug":"cosmos3-inverse-dynamics-prompt-must-be-a-string","errorCode":null,"errorMessage":"Cosmos3 inverse_dynamics prompt must be a string","messagePattern":"Cosmos3 inverse_dynamics prompt must be a string","errorType":"validation","errorClass":"ValueError","httpStatus":400,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py","lineNumber":200,"sourceCode":"        raise ValueError(\n            f\"Cosmos3 action batch size {batch_size} exceeds \"\n            f\"--batching-max-size={max_batch_size}\"\n        )\n    image_path = None if not images else images[0] if batch_size == 1 else images\n\n    domain_id = options.get(\"domain_id\")\n    domain_name = options.get(\"domain_name\")\n    raw_action_dim = options.get(\"raw_action_dim\")\n    if domain_id is None and not domain_name:\n        raise ValueError(\"Cosmos3 action requests require domain_name or domain_id\")\n    if domain_id is not None and not domain_name and raw_action_dim is None:\n        raise ValueError(\"raw_action_dim is required when only domain_id is provided\")\n\n    prompt = observation.get(\"prompt\")\n    if prompt is None:\n        prompt = observation.get(\"task\", \"\")\n    if action_mode != \"policy\" and not isinstance(prompt, str):\n        raise ValueError(\"Cosmos3 inverse_dynamics prompt must be a string\")\n    prompt = _action_prompt(prompt, batch_size)\n    sampling_kwargs = {\n        \"request_id\": payload.get(\"request_id\") or payload.get(\"id\"),\n        \"prompt\": prompt,\n        \"image_path\": image_path,\n        \"video_path\": video_path,\n        \"action_mode\": action_mode,\n        \"domain_id\": domain_id,\n        \"domain_name\": domain_name,\n        \"raw_action_dim\": raw_action_dim,\n        \"action_fps\": options.get(\"action_fps\"),\n        \"action_view_point\": options.get(\"action_view_point\", \"ego_view\"),\n        \"action_normalization\": options.get(\"action_normalization\", \"quantile\"),\n        \"action_stats_path\": server_args.pipeline_config.action_stats_path,\n        \"num_frames\": num_frames,\n        \"fps\": int(options.get(\"fps\", 5)),\n        \"height\": int(options.get(\"height\", 480)),\n        \"width\": int(options.get(\"width\", 832)),","sourceCodeStart":182,"sourceCodeEnd":218,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/entrypoints/action/cosmos3.py#L182-L218","documentation":"In Cosmos3 inverse_dynamics mode the prompt (the described action/task) must be a plain string, not a list. Policy mode allows a list of per-image prompts, but inverse_dynamics processes a single video so a list prompt is rejected via isinstance(prompt, str).","triggerScenarios":"POST /v1/actions with action_mode='inverse_dynamics' and observation.prompt = [\"a\", \"b\"] (or a non-string like an int after the task fallback).","commonSituations":"Reusing a batched policy client payload for inverse_dynamics; prompt falling back to observation.task containing structured data.","solutions":["Pass prompt as a single string for inverse_dynamics requests","If reusing policy code paths, take prompt[0] or join the list when switching modes"],"exampleFix":"# before\n{\"action_mode\": \"inverse_dynamics\", \"observation\": {\"prompt\": [\"lift arm\"]}}\n# after\n{\"action_mode\": \"inverse_dynamics\", \"observation\": {\"prompt\": \"lift arm\"}}","handlingStrategy":"type-guard","validationCode":"if mode == 'inverse_dynamics':\n    assert isinstance(observation.get('prompt', observation.get('task','')), str)","typeGuard":"def is_inverse_prompt_ok(mode, prompt) -> bool:\n    return mode == 'policy' or isinstance(prompt, str)","tryCatchPattern":null,"preventionTips":["Use a single-string prompt for inverse_dynamics","Flatten list prompts when reusing policy clients"],"tags":["cosmos3","inverse-dynamics","prompt-validation","type-validation"],"backgroundTag":"prompt-type-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}