{"record":{"id":"3556171adad1f5e5","repo":"sgl-project/sglang","slug":"out-channels-must-be-divisible-by-tp-size-for-tp-s","errorCode":null,"errorMessage":"out_channels must be divisible by tp_size for TP-sharded output projection, got {arch.out_channels=} {tp_size=}.","messagePattern":"out_channels must be divisible by tp_size for TP-sharded output projection, got (.+?) (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/ltx_2.py","lineNumber":1632,"sourceCode":"                f\"{self.num_attention_heads=} {tp_size=}.\"\n            )\n        if self.audio_num_attention_heads % tp_size != 0:\n            raise ValueError(\n                \"audio_num_attention_heads must be divisible by tp_size, got \"\n                f\"{self.audio_num_attention_heads=} {tp_size=}.\"\n            )\n        if self.hidden_size % tp_size != 0:\n            raise ValueError(\n                \"hidden_size must be divisible by tp_size for TP-sharded projections, got \"\n                f\"{self.hidden_size=} {tp_size=}.\"\n            )\n        if self.audio_hidden_size % tp_size != 0:\n            raise ValueError(\n                \"audio_hidden_size must be divisible by tp_size for TP-sharded projections, got \"\n                f\"{self.audio_hidden_size=} {tp_size=}.\"\n            )\n        if int(arch.out_channels) % tp_size != 0:\n            raise ValueError(\n                \"out_channels must be divisible by tp_size for TP-sharded output projection, got \"\n                f\"{arch.out_channels=} {tp_size=}.\"\n            )\n        if int(arch.audio_out_channels) % tp_size != 0:\n            raise ValueError(\n                \"audio_out_channels must be divisible by tp_size for TP-sharded output projection, got \"\n                f\"{arch.audio_out_channels=} {tp_size=}.\"\n            )\n\n    def __init__(\n        self,\n        config: LTX2Config,\n        hf_config: dict[str, Any],\n        quant_config: QuantizationConfig | None = None,\n    ) -> None:\n        super().__init__(config=config, hf_config=hf_config)\n\n        arch = self.config","sourceCodeStart":1614,"sourceCodeEnd":1650,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/ltx_2.py#L1614-L1650","documentation":"LTX-2 DiT model validates at construction time that the architecture's out_channels is divisible by the tensor-parallel size, because the output projection layer is TP-sharded (column/row parallel split of the linear weight). If out_channels % tp_size != 0, sharding would silently produce mismatched per-rank slice sizes, so the model refuses to build.","triggerScenarios":"Instantiating the LTX-2 model (or its runtime wrapper) with --tp / tp_size set to a value that does not divide arch.out_channels, e.g. out_channels=128 with tp_size=6, or loading a checkpoint config whose out_channels is an odd number while running multi-GPU TP.","commonSituations":"Choosing a large TP degree (e.g. 5, 6, 7) to fill available GPUs without checking channel divisibility; editing the model config's out_channels; using a nonstandard checkpoint variant with unusual channel counts.","solutions":["Pick a tp_size that divides arch.out_channels (powers of two and small factors of the channel count, e.g. 1, 2, 4, 8)","Print/check arch.out_channels from the loaded config before launching and factorize it to find valid TP degrees","If the divisibility can never be satisfied for your hardware, run with tp_size=1 or shard a different dimension (e.g. use DP instead of TP)"],"exampleFix":"# before: tp_size=6, out_channels=128 -> ValueError\nmodel = Ltx2Model(arch, tp_size=6)\n\n# after\nassert arch.out_channels % tp_size == 0\nmodel = Ltx2Model(arch, tp_size=4)","handlingStrategy":"validation","validationCode":"import math\nvalid_tp = [t for t in range(1, 9) if int(arch.out_channels) % t == 0]\nassert tp_size in valid_tp, f'tp_size must be one of {valid_tp}'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Factor out_channels before choosing TP degree","Add a startup assert on divisibility for all sharded dims","Prefer power-of-two TP degrees for channel counts that are powers of two"],"tags":["tp-sharding","divisibility","ltx-2","config-validation","tensor-parallel"],"backgroundTag":"tensor-parallel-size-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}