{"record":{"id":"5d5ba39ef4e18721","repo":"sgl-project/sglang","slug":"sana-wm-does-not-support-temporal-sequence-paralle","errorCode":null,"errorMessage":"SANA-WM does not support temporal sequence parallelism yet. Stage-1 GDN/GLUMBConvTemp span frames and require halo/state exchange before latents can be sharded. Use --num-gpus with FSDP/CFG parallelism instead of --sp-degree {sp_degree}.","messagePattern":"SANA-WM does not support temporal sequence parallelism yet\\. Stage-1 GDN/GLUMBConvTemp span frames and require halo/state exchange before latents can be sharded\\. Use --num-gpus with FSDP/CFG parallelism instead of --sp-degree (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/pipelines/sana_wm_pipeline.py","lineNumber":75,"sourceCode":"        \"tokenizer\",\n        \"vae\",\n        \"transformer\",\n        \"scheduler\",\n    ]\n\n    @staticmethod\n    def _validate_parallelism_args(server_args: ServerArgs) -> None:\n        tp_size = getattr(server_args, \"tp_size\", 1) or 1\n        if tp_size != 1:\n            raise ValueError(\n                \"SANA-WM does not support tensor parallelism yet. \"\n                \"Use --num-gpus with FSDP/CFG parallelism instead of \"\n                f\"--tp-size {tp_size}.\"\n            )\n\n        sp_degree = getattr(server_args, \"sp_degree\", 1) or 1\n        if sp_degree != 1:\n            raise ValueError(\n                \"SANA-WM does not support temporal sequence parallelism yet. \"\n                \"Stage-1 GDN/GLUMBConvTemp span frames and require halo/state \"\n                \"exchange before latents can be sharded. Use --num-gpus with \"\n                \"FSDP/CFG parallelism instead of \"\n                f\"--sp-degree {sp_degree}.\"\n            )\n\n    def create_pipeline_stages(self, server_args: ServerArgs):\n        self._validate_parallelism_args(server_args)\n        self.add_stage(InputValidationStage())\n\n        self.add_stage(\n            SanaWMTextEncodingStage(\n                text_encoders=[self.get_module(\"text_encoder\")],\n                tokenizers=[self.get_module(\"tokenizer\")],\n            ),\n            \"prompt_encoding_stage\",\n        )","sourceCodeStart":57,"sourceCodeEnd":93,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/pipelines/sana_wm_pipeline.py#L57-L93","documentation":"Raised by SANA-WM's _validate_parallelism_args when server_args.sp_degree != 1. Stage-1 GDN and GLUMBConvTemp layers operate across temporal frames and would need halo/state exchange before latents can be sharded, so temporal sequence parallelism is rejected until implemented.","triggerScenarios":"Launching SANA-WM with --sp-degree 2 or higher; configs that enable sequence parallelism by default on multi-GPU nodes; tests directly exercising _validate_parallelism_args with sp_degree > 1.","commonSituations":"Enabling SP for long-video workloads assuming it is supported like in other video pipelines; presets carrying sp_degree from a different model's tuning; copy-pasted multi-GPU flags.","solutions":["Remove --sp-degree (or set it to 1) and scale with --num-gpus + FSDP/CFG parallelism","Adjust shared launch templates so sp_degree is not applied to SANA-WM","Track upstream for temporal-SP support (halo/state exchange) before retrying"],"exampleFix":"# before\n--sp-degree 4 --model sana-wm\n\n# after\n--sp-degree 1 --num-gpus 4 --model sana-wm","handlingStrategy":"validation","validationCode":"sp = getattr(server_args, 'sp_degree', 1) or 1\nif sp != 1:\n    raise SystemExit('SANA-WM: temporal SP unsupported; set --sp-degree 1 and use --num-gpus')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Strip sp_degree from configs for pipelines without temporal SP support","Track upstream support before enabling sequence parallelism"],"tags":["sana-wm","sequence-parallelism","unsupported-feature","sglang"],"backgroundTag":"invalid-parallelism-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}