{"record":{"id":"40b1c0a9bbd5ed07","repo":"hiyouga/LlamaFactory","slug":"ep-size-must-be-positive-got-ep-size","errorCode":null,"errorMessage":"ep_size must be positive, got {ep_size}.","messagePattern":"ep_size must be positive, got (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/trainer_plugins/distributed/fsdpturbo.py","lineNumber":60,"sourceCode":"        self.efsdp_size = 1\n        self.edp_size = 1\n        self.expert_mesh: DeviceMesh | None = None\n        self.edp_mesh: DeviceMesh | None = None\n        self.efsdp_mesh: DeviceMesh | None = None\n        self.ep_mesh: DeviceMesh | None = None\n        self.expert_cp_mesh: DeviceMesh | None = None\n\n    @property\n    def initialized(self) -> bool:\n        return self._initialized\n\n    def initialize(self, dist_interface: DistributedInterface, dist_config: dict) -> None:\n        dp_size = dist_interface.get_world_size(Dim.DP)\n        cp_size = dist_interface.strategy.cp_size\n        ep_size = int(dist_config.get(\"ep_size\", 1))\n\n        if ep_size < 1:\n            raise ValueError(f\"ep_size must be positive, got {ep_size}.\")\n        if dp_size % ep_size != 0:\n            raise ValueError(f\"dp_size must be divisible by ep_size, got {dp_size} % {ep_size} != 0.\")\n\n        topology = (dp_size, cp_size, ep_size)\n        if self._initialized:\n            current_topology = (self.dp_size, self.cp_size, self.ep_size)\n            if topology != current_topology:\n                raise RuntimeError(\n                    f\"FSDPTurbo parallel state is already initialized with {current_topology}, got {topology}.\"\n                )\n            return\n\n        self.dp_size = dp_size\n        self.cp_size = cp_size\n        self.ep_size = ep_size\n\n        if ep_size > 1:\n            self.efsdp_size = dp_size // ep_size","sourceCodeStart":42,"sourceCodeEnd":78,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/trainer_plugins/distributed/fsdpturbo.py#L42-L78","documentation":"FSDPTurbo's parallel-state initialization reads ep_size (expert parallel size) from dist_config, defaulting to 1. Expert parallelism partitions MoE experts across ranks, so ep_size must be a positive integer; values of 0 or negatives make the mesh shape invalid and are rejected immediately.","triggerScenarios":"dist_config['ep_size'] set to 0 or a negative number (or a string/malformed value that int() maps to <= 0).","commonSituations":"User disables EP by setting ep_size: 0 instead of omitting it or using 1; YAML math/env interpolation produces 0; copy-paste from a template with a placeholder value.","solutions":["Set ep_size to a positive divisor of dp_size (e.g. 2, 4), or remove the key / set 1 for no expert parallelism","Check the resolved value (int(dist_config.get('ep_size', 1))) in a dry run"],"exampleFix":"# before\ndist_config = {\"ep_size\": 0}\n\n# after\ndist_config = {\"ep_size\": 1}  # or omit the key","handlingStrategy":"validation","validationCode":"ep = int(dist_config.get(\"ep_size\", 1))\nassert ep >= 1, f\"ep_size must be >= 1, got {ep}\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use 1 or omit ep_size to disable expert parallelism, never 0","Validate parallel config integers before launching distributed jobs"],"tags":["fsdpturbo","expert-parallel","distributed","configuration"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}