{"record":{"id":"bf01e516f0a06e07","repo":"sgl-project/sglang","slug":"cosmos3-action-generation-does-not-support-sequenc","errorCode":null,"errorMessage":"Cosmos3 action generation does not support sequence parallelism yet","messagePattern":"Cosmos3 action generation does not support sequence parallelism yet","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/cosmos3video.py","lineNumber":1497,"sourceCode":"            (video_pred, ...) with extra tensors when action/sound are active.\n        \"\"\"\n        if text_ids is None or text_mask is None:\n            raise ValueError(\"Cosmos3 requires text_ids and text_mask to be passed\")\n\n        batch_size, C, T, H, W = hidden_states.shape\n        Hp, Wp, _, _ = self._pad_to_patch_size(H, W)\n        if max_text_seq_len is None:\n            max_text_seq_len = int(text_mask.sum(dim=1).max().item())\n        if max_text_seq_len < text_ids.shape[1]:\n            text_ids = text_ids[:, :max_text_seq_len]\n            text_mask = text_mask[:, :max_text_seq_len]\n\n        sound_frames = sound_latents.shape[-1] if sound_latents is not None else 0\n\n        action_frames = 0\n        if action_latents is not None:\n            if self.sp_size > 1:\n                raise NotImplementedError(\n                    \"Cosmos3 action generation does not support sequence parallelism yet\"\n                )\n            action_frames = action_latents.shape[1]\n            if action_domain_ids is None:\n                action_domain_ids = torch.zeros(\n                    action_latents.shape[0],\n                    dtype=torch.long,\n                    device=action_latents.device,\n                )\n\n        extra_frames = action_frames + sound_frames\n        sequence_shard_enabled = self.sp_size > 1\n\n        # Add timestep embedding (computed in float32 for numerical stability, then cast back)\n        time_embed = self.time_embedder(timestep.float())\n        time_embed = time_embed.to(\n            hidden_states.dtype\n        )  # Cast to match hidden_gen dtype","sourceCodeStart":1479,"sourceCodeEnd":1515,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/cosmos3video.py#L1479-L1515","documentation":"Cosmos3's action-generation branch (action_latents provided) is not implemented for sequence parallelism. If action latents are supplied while sp_size > 1, forward raises NotImplementedError rather than silently producing wrong sharded results.","triggerScenarios":"Launching with sequence parallelism (--sp / sp_size > 1) and then requesting action generation by passing action_latents to the Cosmos3 forward pass.","commonSituations":"Enabling SP for throughput on large-resolution video and then trying an action-conditioned generation mode; the flag combination passes config validation but hits the unimplemented code path at runtime.","solutions":["Disable sequence parallelism (set sp_size=1 / drop the SP flag) when action generation is required","Keep action workloads on a separate deployment without SP enabled","Track upstream sglang for the SP implementation of action generation before combining the two"],"exampleFix":"# before\nserver_args.enable_sp = True   # + action generation request\n# after\nserver_args.enable_sp = False  # action generation requires sp_size == 1","handlingStrategy":"validation","validationCode":"if action_latents is not None:\n    assert get_sp_world_size() == 1 or sp_size == 1, \"action generation requires sp_size == 1\"","typeGuard":"def action_gen_supported(sp_size: int) -> bool:\n    return sp_size == 1","tryCatchPattern":"try:\n    out = dit(..., action_latents=action_latents)\nexcept NotImplementedError:\n    out = dit_no_sp(...)  # rerun with sp disabled","preventionTips":["Gate feature flags against a capability matrix (SP x action) at server startup","Keep action-generation workloads on non-SP deployments"],"tags":["sglang","cosmos3","action-generation","sequence-parallel","unsupported-feature"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}