{"record":{"id":"2abc3e43af242ad6","repo":"sgl-project/sglang","slug":"invalid-tp-size-expected-tp-size-1","errorCode":null,"errorMessage":"Invalid {tp_size=}. Expected tp_size >= 1.","messagePattern":"Invalid (.+?)\\. Expected tp_size >= 1\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/ltx_2.py","lineNumber":758,"sourceCode":"    ) -> None:\n        super().__init__()\n\n        is_cross_attention = context_dim is not None\n        self.query_dim = int(query_dim)\n        self.context_dim = int(query_dim if context_dim is None else context_dim)\n        self.heads = int(heads)\n        self.dim_head = int(dim_head)\n        self.inner_dim = self.heads * self.dim_head\n        self.norm_eps = float(norm_eps)\n        self.qk_norm = bool(qk_norm)\n        self.use_local_attention = bool(use_local_attention)\n        self.apply_gated_attention = bool(apply_gated_attention)\n        self.enable_packed_qkv_input_a2a = bool(enable_packed_qkv_input_a2a)\n        self.prefix = prefix\n\n        tp_size = get_tp_world_size()\n        if tp_size <= 0:\n            raise ValueError(f\"Invalid {tp_size=}. Expected tp_size >= 1.\")\n        if self.heads % tp_size != 0:\n            raise ValueError(\n                f\"LTX2Attention requires heads divisible by tp_size, got \"\n                f\"{self.heads=} {tp_size=}.\"\n            )\n        if self.inner_dim % tp_size != 0:\n            # This should follow from heads % tp_size, but keep explicit for clarity.\n            raise ValueError(\n                f\"LTX2Attention requires inner_dim divisible by tp_size, got \"\n                f\"{self.inner_dim=} {tp_size=}.\"\n            )\n        self.local_heads = self.heads // tp_size\n\n        self.to_q = ColumnParallelLinear(\n            self.query_dim,\n            self.inner_dim,\n            bias=True,\n            gather_output=False,","sourceCodeStart":740,"sourceCodeEnd":776,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/ltx_2.py#L740-L776","documentation":"LTX2Attention derives local head/inner-dim splits from the tensor-parallel world size and sanity-checks that it is a positive integer. tp_size <= 0 indicates a broken/uninitialized parallel group (get_tp_world_size() returned 0 or negative), which would corrupt all sharding math.","triggerScenarios":"Constructing LTX2Attention before TP groups are initialized (e.g. in a unit test or offline script without initializing the distributed backend), where get_tp_world_size() returns 0.","commonSituations":"Importing/instantiating the model outside the server launch flow; a worker where the TP group init failed silently; mocking distributed helpers with wrong defaults.","solutions":["Initialize the distributed/TP groups before building the model (use the standard launcher)","In tests, mock get_tp_world_size to return 1 after initializing a single-process group","If it persists in serving, check worker startup logs for failed group initialization"],"exampleFix":"# before (test)\nlayer = LTX2Attention(...)   # TP group never initialized -> tp_size=0\n\n# after (test)\nimport torch.distributed as dist\ndist.init_process_group(\"nccl\", rank=0, world_size=1)\nlayer = LTX2Attention(...)","handlingStrategy":"validation","validationCode":"tp = get_tp_world_size()\\nassert tp >= 1, f'distributed groups not initialized (tp_size={tp}); init process group first'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always init process groups before building TP modules","In tests, use a 1-rank init_process_group","Check worker startup logs for group init failures"],"tags":["parallelism","tensor-parallel","init-order","distributed","ltx2"],"backgroundTag":"distributed-group-not-initialized","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}