{"record":{"id":"318c390ee3b3f06d","repo":"hiyouga/LlamaFactory","slug":"world-size-helper-get-world-size-must-be-div-318c39","errorCode":null,"errorMessage":"world_size ({helper.get_world_size()}) must be divisible by cp_size ({self.cp_size}).","messagePattern":"world_size \\((.+?)\\) must be divisible by cp_size \\((.+?)\\)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/accelerator/interface.py","lineNumber":91,"sourceCode":"            self.mp_shard_size = 1\n        elif self.mp_shard_size is None:\n            if helper.get_world_size() % self.mp_replicate_size != 0:\n                raise ValueError(\n                    f\"world_size ({helper.get_world_size()}) must be divisible by \"\n                    f\"mp_replicate_size ({self.mp_replicate_size}).\"\n                )\n            self.mp_shard_size = helper.get_world_size() // self.mp_replicate_size\n        elif self.mp_replicate_size * self.mp_shard_size != helper.get_world_size():\n            raise ValueError(\n                f\"mp_replicate_size * mp_shard_size must equal to world_size, \"\n                f\"got {self.mp_replicate_size} * {self.mp_shard_size} != {helper.get_world_size()}.\"\n            )\n\n        if not helper.is_distributed():\n            self.dp_size = 1\n        elif self.dp_size is None:\n            if helper.get_world_size() % self.cp_size != 0:\n                raise ValueError(\n                    f\"world_size ({helper.get_world_size()}) must be divisible by cp_size ({self.cp_size}).\"\n                )\n            self.dp_size = helper.get_world_size() // self.cp_size\n        elif self.dp_size * self.cp_size != helper.get_world_size():\n            raise ValueError(\n                f\"dp_size * cp_size must equal to world_size, \"\n                f\"got {self.dp_size} * {self.cp_size} != {helper.get_world_size()}.\"\n            )\n\n    @property\n    def model_mesh_shape(self) -> tuple[int, int]:\n        \"\"\"Model parallel mesh shape.\"\"\"\n        return (self.mp_replicate_size, self.mp_shard_size)\n\n    @property\n    def model_mesh_dim_names(self) -> tuple[str, str]:\n        \"\"\"Model parallel mesh dimension names.\"\"\"\n        return (Dim.MP_REPLICATE.value, Dim.MP_SHARD.value)","sourceCodeStart":73,"sourceCodeEnd":109,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/accelerator/interface.py#L73-L109","documentation":"Raised in `DistributedConfig.__post_init__` when `dp_size` is unset in a distributed run and world size is not divisible by `cp_size` (context parallel size). The data-parallel dimension is derived as `world_size // cp_size`, so context parallelism must partition the ranks evenly.","triggerScenarios":"Setting `cp_size: 3` (or any non-divisor) in the v1 training args while launching with e.g. 4 or 8 ranks and leaving `dp_size` unset.","commonSituations":"Enabling context/sequence parallelism for long-context training with a `cp_size` copied from a different GPU topology; using `cp_size` larger than world size; odd `cp_size` on power-of-two GPU counts.","solutions":["Use a `cp_size` that divides world size evenly (1, 2, 4, 8 on 8 ranks)","If you need an unusual split, set `dp_size` explicitly so the product check applies instead","Remember `cp_size` cannot exceed world size; reduce it if you shrank the job"],"exampleFix":"# before: 8 ranks, cp=3\ntraining:\n  cp_size: 3   # 8 % 3 != 0\n\n# after\ntraining:\n  cp_size: 4   # 8 % 4 == 0, dp_size derived as 2","handlingStrategy":"validation","validationCode":"def validate_cp(world_size: int, cp_size: int) -> None:\n    if world_size % cp_size != 0:\n        raise SystemExit(f\"cp_size={cp_size} must divide world_size={world_size}\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Restrict cp_size to powers of two on typical GPU counts","Add a config linter that checks cp_size <= world_size and divisibility","Log world_size in job startup logs to debug mismatches quickly"],"tags":["distributed-training","v1","context-parallelism","configuration"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}