{"record":{"id":"f1f1155cdd34be31","repo":"sgl-project/sglang","slug":"sana-wm-does-not-support-tensor-parallelism-yet-u","errorCode":null,"errorMessage":"SANA-WM does not support tensor parallelism yet. Use --num-gpus with FSDP/CFG parallelism instead of --tp-size {tp_size}.","messagePattern":"SANA-WM does not support tensor parallelism yet\\. Use --num-gpus with FSDP/CFG parallelism instead of --tp-size (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/pipelines/sana_wm_pipeline.py","lineNumber":67,"sourceCode":"    \"\"\"SANA-WM TI2V pipeline (single-stage).\"\"\"\n\n    pipeline_name = \"SanaWMPipeline\"\n    pipeline_config_cls = SanaWMPipelineConfig\n    sampling_params_cls = SanaWMSamplingParams\n\n    _required_config_modules = [\n        \"text_encoder\",\n        \"tokenizer\",\n        \"vae\",\n        \"transformer\",\n        \"scheduler\",\n    ]\n\n    @staticmethod\n    def _validate_parallelism_args(server_args: ServerArgs) -> None:\n        tp_size = getattr(server_args, \"tp_size\", 1) or 1\n        if tp_size != 1:\n            raise ValueError(\n                \"SANA-WM does not support tensor parallelism yet. \"\n                \"Use --num-gpus with FSDP/CFG parallelism instead of \"\n                f\"--tp-size {tp_size}.\"\n            )\n\n        sp_degree = getattr(server_args, \"sp_degree\", 1) or 1\n        if sp_degree != 1:\n            raise ValueError(\n                \"SANA-WM does not support temporal sequence parallelism yet. \"\n                \"Stage-1 GDN/GLUMBConvTemp span frames and require halo/state \"\n                \"exchange before latents can be sharded. Use --num-gpus with \"\n                \"FSDP/CFG parallelism instead of \"\n                f\"--sp-degree {sp_degree}.\"\n            )\n\n    def create_pipeline_stages(self, server_args: ServerArgs):\n        self._validate_parallelism_args(server_args)\n        self.add_stage(InputValidationStage())","sourceCodeStart":49,"sourceCodeEnd":85,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/pipelines/sana_wm_pipeline.py#L49-L85","documentation":"Raised by SANA-WM's _validate_parallelism_args during create_pipeline_stages when server_args.tp_size != 1. The watermarker/world-model pipeline has no tensor-parallel implementation yet; the message directs users to multi-GPU FSDP/CFG parallelism via --num-gpus instead.","triggerScenarios":"Launching the SANA-WM pipeline with --tp-size 2/4/8; cluster templates that default tp_size > 1 for all models; calling _validate_parallelism_args directly with a TP-configured ServerArgs (as the tests do).","commonSituations":"Shared GPU-cluster launch presets with TP baked in; assuming all sglang pipelines support TP; migrating a TP deployment to SANA-WM.","solutions":["Set --tp-size 1 (or omit it) and use --num-gpus with FSDP/CFG parallelism for multi-GPU scaling","Update cluster/Helm templates to strip TP flags for SANA-WM jobs","Watch for upstream TP support before re-enabling the flag"],"exampleFix":"# before\n--tp-size 4 --model sana-wm\n\n# after\n--tp-size 1 --num-gpus 4 --model sana-wm","handlingStrategy":"validation","validationCode":"tp = getattr(server_args, 'tp_size', 1) or 1\nif tp != 1:\n    raise SystemExit('SANA-WM: set --tp-size 1; scale with --num-gpus/FSDP instead')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Per-pipeline validation of parallelism flags at config load","Do not assume TP support across all multimodal pipelines"],"tags":["sana-wm","tensor-parallelism","unsupported-feature","sglang"],"backgroundTag":"invalid-parallelism-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}