sgl-project/sglang · error · NotImplementedError

Cosmos3 action generation does not support sequence parallel

Error message

Cosmos3 action generation does not support sequence parallelism yet

What it means

Cosmos3's action-generation branch (action_latents provided) is not implemented for sequence parallelism. If action latents are supplied while sp_size > 1, forward raises NotImplementedError rather than silently producing wrong sharded results.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/cosmos3video.py:1497

            (video_pred, ...) with extra tensors when action/sound are active.
        """
        if text_ids is None or text_mask is None:
            raise ValueError("Cosmos3 requires text_ids and text_mask to be passed")

        batch_size, C, T, H, W = hidden_states.shape
        Hp, Wp, _, _ = self._pad_to_patch_size(H, W)
        if max_text_seq_len is None:
            max_text_seq_len = int(text_mask.sum(dim=1).max().item())
        if max_text_seq_len < text_ids.shape[1]:
            text_ids = text_ids[:, :max_text_seq_len]
            text_mask = text_mask[:, :max_text_seq_len]

        sound_frames = sound_latents.shape[-1] if sound_latents is not None else 0

        action_frames = 0
        if action_latents is not None:
            if self.sp_size > 1:
                raise NotImplementedError(
                    "Cosmos3 action generation does not support sequence parallelism yet"
                )
            action_frames = action_latents.shape[1]
            if action_domain_ids is None:
                action_domain_ids = torch.zeros(
                    action_latents.shape[0],
                    dtype=torch.long,
                    device=action_latents.device,
                )

        extra_frames = action_frames + sound_frames
        sequence_shard_enabled = self.sp_size > 1

        # Add timestep embedding (computed in float32 for numerical stability, then cast back)
        time_embed = self.time_embedder(timestep.float())
        time_embed = time_embed.to(
            hidden_states.dtype
        )  # Cast to match hidden_gen dtype

View on GitHub (pinned to 0132848349)

Solutions

1. Disable sequence parallelism (set sp_size=1 / drop the SP flag) when action generation is required
2. Keep action workloads on a separate deployment without SP enabled
3. Track upstream sglang for the SP implementation of action generation before combining the two

Example fix

# before
server_args.enable_sp = True   # + action generation request
# after
server_args.enable_sp = False  # action generation requires sp_size == 1

if action_latents is not None:
    assert get_sp_world_size() == 1 or sp_size == 1, "action generation requires sp_size == 1"

def action_gen_supported(sp_size: int) -> bool:
    return sp_size == 1

try:
    out = dit(..., action_latents=action_latents)
except NotImplementedError:
    out = dit_no_sp(...)  # rerun with sp disabled

When it happens

Trigger: Launching with sequence parallelism (--sp / sp_size > 1) and then requesting action generation by passing action_latents to the Cosmos3 forward pass.

Common situations: Enabling SP for throughput on large-resolution video and then trying an action-conditioned generation mode; the flag combination passes config validation but hits the unimplemented code path at runtime.

Related errors

• Cosmos3CausalAttention requires num_attention_heads divisibl
• Cosmos3CausalAttention requires num_key_value_heads divisibl
• Cosmos3CrossAttention requires num_attention_heads divisible
• Cosmos3CrossAttention requires num_key_value_heads divisible
• Cosmos3 requires text_ids and text_mask to be passed

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/bf01e516f0a06e07. Report an issue: GitHub.