sgl-project/sglang · error · NotImplementedError

Cosmos3 action generation does not support CFG parallel yet

Error message

Cosmos3 action generation does not support CFG parallel yet

What it means

Cosmos3 raises NotImplementedError when action generation is combined with CFG parallelism (server_args.enable_cfg_parallel with guidance_scale > 1). The separate conditional/unconditional CFG-parallel batch streams are not wired up for the 3D action tensors, unlike video.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/cosmos3.py:1089

                image_kwargs={},
                pos_cond_kwargs={
                    "text_ids": cond_text_ids,
                    "text_mask": cond_text_mask,
                    "fps": fps,
                },
                neg_cond_kwargs={
                    "text_ids": uncond_text_ids,
                    "text_mask": uncond_text_mask,
                    "fps": fps,
                },
                guidance=None,
            )

        do_cfg = guidance_scale > 1.0

        enable_cfg_parallel = server_args.enable_cfg_parallel and do_cfg
        if action_latents is not None and enable_cfg_parallel:
            raise NotImplementedError(
                "Cosmos3 action generation does not support CFG parallel yet"
            )

        # Use separate scheduler instances for action/sound: UniPC keeps a
        # per-call output history sized to the last sample, so video (5D),
        # action (3D), and sound (3D) steps must not share state.
        sound_scheduler = None
        if sound_latents is not None:
            sound_scheduler = copy.deepcopy(self.scheduler)
            sound_scheduler.set_timesteps(len(timesteps), device=timesteps.device)
        action_scheduler = None
        if action_latents is not None:
            action_scheduler = copy.deepcopy(self.scheduler)
            action_scheduler.set_timesteps(len(timesteps), device=timesteps.device)
        cfg_rank = get_classifier_free_guidance_rank() if enable_cfg_parallel else 0
        cfg_world_size = (
            get_classifier_free_guidance_world_size() if enable_cfg_parallel else 1
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Disable --enable-cfg-parallel for action-generation workloads
2. Or set guidance_scale <= 1.0 so do_cfg (and thus enable_cfg_parallel) is False
3. Keep per-modality launch scripts so video-only perf flags don't leak into action runs

Example fix

# before
--enable-cfg-parallel --guidance-scale 4.0  # action request -> NotImplementedError
# after
--guidance-scale 4.0  # CFG runs serially, works for actions

do_cfg = guidance_scale > 1.0
assert not (server_args.enable_cfg_parallel and do_cfg and uses_action_latents), "CFG parallel unsupported for action generation"

def cfg_parallel_ok(server_args, guidance_scale: float, has_actions: bool) -> bool:
    return not (server_args.enable_cfg_parallel and guidance_scale > 1.0 and has_actions)

When it happens

Trigger: Launching with --enable-cfg-parallel and guidance_scale > 1.0, then sending a request with action latents (DataType.ACTION path / action_latents not None) into forward.

Common situations: Copying a tuned video-generation launch config (CFG parallel on) to an action-dynamics workload; enabling global perf flags without checking modality support matrices.

Related errors

• Cosmos3 action generation does not support sequence parallel
• action_mode is set but the loaded Cosmos3 checkpoint has no
• domain_id must be non-negative, got {domain_id}
• Unknown action domain name {domain_name!r}. Valid names: {so
• Cosmos3 action generation requires --domain-id or --domain-n

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/419afdba69bf5c56. Report an issue: GitHub.