sgl-project/sglang · error · ValueError

SANA-WM does not support temporal sequence parallelism yet.

Error message

SANA-WM does not support temporal sequence parallelism yet. Stage-1 GDN/GLUMBConvTemp span frames and require halo/state exchange before latents can be sharded. Use --num-gpus with FSDP/CFG parallelism instead of --sp-degree {sp_degree}.

What it means

Raised by SANA-WM's _validate_parallelism_args when server_args.sp_degree != 1. Stage-1 GDN and GLUMBConvTemp layers operate across temporal frames and would need halo/state exchange before latents can be sharded, so temporal sequence parallelism is rejected until implemented.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines/sana_wm_pipeline.py:75

        "tokenizer",
        "vae",
        "transformer",
        "scheduler",
    ]

    @staticmethod
    def _validate_parallelism_args(server_args: ServerArgs) -> None:
        tp_size = getattr(server_args, "tp_size", 1) or 1
        if tp_size != 1:
            raise ValueError(
                "SANA-WM does not support tensor parallelism yet. "
                "Use --num-gpus with FSDP/CFG parallelism instead of "
                f"--tp-size {tp_size}."
            )

        sp_degree = getattr(server_args, "sp_degree", 1) or 1
        if sp_degree != 1:
            raise ValueError(
                "SANA-WM does not support temporal sequence parallelism yet. "
                "Stage-1 GDN/GLUMBConvTemp span frames and require halo/state "
                "exchange before latents can be sharded. Use --num-gpus with "
                "FSDP/CFG parallelism instead of "
                f"--sp-degree {sp_degree}."
            )

    def create_pipeline_stages(self, server_args: ServerArgs):
        self._validate_parallelism_args(server_args)
        self.add_stage(InputValidationStage())

        self.add_stage(
            SanaWMTextEncodingStage(
                text_encoders=[self.get_module("text_encoder")],
                tokenizers=[self.get_module("tokenizer")],
            ),
            "prompt_encoding_stage",
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Remove --sp-degree (or set it to 1) and scale with --num-gpus + FSDP/CFG parallelism
2. Adjust shared launch templates so sp_degree is not applied to SANA-WM
3. Track upstream for temporal-SP support (halo/state exchange) before retrying

Example fix

# before
--sp-degree 4 --model sana-wm

# after
--sp-degree 1 --num-gpus 4 --model sana-wm

sp = getattr(server_args, 'sp_degree', 1) or 1
if sp != 1:
    raise SystemExit('SANA-WM: temporal SP unsupported; set --sp-degree 1 and use --num-gpus')

When it happens

Trigger: Launching SANA-WM with --sp-degree 2 or higher; configs that enable sequence parallelism by default on multi-GPU nodes; tests directly exercising _validate_parallelism_args with sp_degree > 1.

Common situations: Enabling SP for long-video workloads assuming it is supported like in other video pipelines; presets carrying sp_degree from a different model's tuning; copy-pasted multi-GPU flags.

Related errors

• SANA-WM does not support tensor parallelism yet. Use --num-g
• Krea-2 sequence parallelism does not support ragged/padded m
• LTX-2 SP time-sharding for packed token latents currently re
• Expected {seq_len=} > 0 for packed token latents.
• Cosmos3 action generation does not support sequence parallel

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5d5ba39ef4e18721. Report an issue: GitHub.