{"record":{"id":"40559d142d80148c","repo":"sgl-project/sglang","slug":"cosmos3-rollout-supports-t2v-t2i-only-i2v-v2v-con","errorCode":null,"errorMessage":"Cosmos3 rollout supports T2V/T2I only; I2V/V2V conditioned-frame re-blending breaks the Gaussian transition assumption of the SDE log-prob math.","messagePattern":"Cosmos3 rollout supports T2V/T2I only; I2V/V2V conditioned-frame re-blending breaks the Gaussian transition assumption of the SDE log-prob math\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/cosmos3.py","lineNumber":1058,"sourceCode":"        generator = batch.generator\n        if generator is None and batch.seed is not None:\n            generator = torch.Generator(device=latents.device).manual_seed(batch.seed)\n\n        cond_text_ids = batch.extra[\"cond_text_ids\"]\n        cond_text_mask = batch.extra[\"cond_text_mask\"]\n        uncond_text_ids = batch.extra[\"uncond_text_ids\"]\n        uncond_text_mask = batch.extra[\"uncond_text_mask\"]\n        video_shape = batch.extra[\"video_shape\"]\n        fps = batch.extra.get(\"fps\", 24.0)\n        velocity_mask = batch.extra.get(\"velocity_mask\")\n        condition_latents = batch.extra.get(\"condition_latents\")\n        guidance_interval = getattr(batch.sampling_params, \"guidance_interval\", None)\n\n        # Rollout requests carry a per-request scheduler bound by the timestep stage.\n        scheduler = batch.scheduler if batch.scheduler is not None else self.scheduler\n        if batch.rollout:\n            if velocity_mask is not None or condition_latents is not None:\n                raise ValueError(\n                    \"Cosmos3 rollout supports T2V/T2I only; I2V/V2V \"\n                    \"conditioned-frame re-blending breaks the Gaussian \"\n                    \"transition assumption of the SDE log-prob math.\"\n                )\n            if action_latents is not None or sound_latents is not None:\n                raise ValueError(\n                    \"Cosmos3 rollout does not support action/sound modalities.\"\n                )\n            self._maybe_prepare_rollout(batch)\n            self._maybe_init_denoising_env_collection(\n                batch=batch,\n                pipeline_config=server_args.pipeline_config,\n                image_kwargs={},\n                pos_cond_kwargs={\n                    \"text_ids\": cond_text_ids,\n                    \"text_mask\": cond_text_mask,\n                    \"fps\": fps,\n                },","sourceCodeStart":1040,"sourceCodeEnd":1076,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/cosmos3.py#L1040-L1076","documentation":"During rollout (RL sampling) the Cosmos3 stage only supports text-to-video/text-to-image. Requests with velocity_mask or condition_latents (I2V/V2V) would require re-blending conditioned frames between SDE steps, which invalidates the Gaussian transition assumption used in the SDE log-probability computation, so the stage refuses them.","triggerScenarios":"Setting batch.rollout=True (rollout/sampling mode) on a request that also supplies condition_latents (image-conditioned video) or velocity_mask; typically triggered by an RL rollout engine feeding I2V/V2V tasks to the pipeline.","commonSituations":"Pointing an RL training loop's Cosmos3 rollout worker at an image-conditioned checkpoint or dataset; mixing T2V and I2V prompts in a rollout batch without filtering.","solutions":["Filter rollout requests to T2V/T2I only (drop image/video conditioning inputs before setting rollout=True)","If I2V rollouts are required, use a different stage/model that supports conditioned-frame SDE log-probs","Gate in the request builder: assert condition inputs are None when rollout is requested"],"exampleFix":"# before\nbatch.rollout = True  # request also has condition_latents set\n# after\nassert batch.condition_latents is None and batch.velocity_mask is None, \"T2V only for rollout\"\nbatch.rollout = True","handlingStrategy":"validation","validationCode":"if batch.rollout:\n    assert batch.condition_latents is None and getattr(batch, \"velocity_mask\", None) is None, \"rollout is T2V/T2I only\"","typeGuard":"def is_rollout_safe(batch) -> bool:\n    return batch.condition_latents is None and getattr(batch, \"velocity_mask\", None) is None","tryCatchPattern":"try:\n    stage.forward(batch)\nexcept ValueError as e:\n    if \"T2V/T2I only\" in str(e):\n        strip_conditioning(batch)  # or reroute request\n    else:\n        raise","preventionTips":["Filter conditioned-frame samples out of rollout datasets","Document the T2V-only constraint next to rollout config in RL training code"],"tags":["cosmos3","rollout","i2v","sde","rl-sampling"],"backgroundTag":"unsupported-operation-mode","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}