{"record":{"id":"4441f3c2dfc69291","repo":"hiyouga/LlamaFactory","slug":"pipeline-model-parallel-size-must-be-1","errorCode":null,"errorMessage":"`pipeline_model_parallel_size` must be >= 1.","messagePattern":"`pipeline_model_parallel_size` must be >= 1\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/megatron_bridge_args.py","lineNumber":156,"sourceCode":"    export_hf_on_finish: bool = field(\n        default=False,\n        metadata={\"help\": \"Whether to export the final checkpoint to Hugging Face format after training.\"},\n    )\n    extra_config: Optional[str] = field(\n        default=None,\n        metadata={\n            \"help\": (\n                \"Optional JSON string or path to a JSON file with extra Megatron Bridge model/training overrides. \"\n                \"Dot-paths are supported (e.g. train.train_iters or checkpoint.save_interval).\"\n            )\n        },\n    )\n\n    def __post_init__(self) -> None:\n        if self.tensor_model_parallel_size < 1:\n            raise ValueError(\"`tensor_model_parallel_size` must be >= 1.\")\n        if self.pipeline_model_parallel_size < 1:\n            raise ValueError(\"`pipeline_model_parallel_size` must be >= 1.\")\n        if self.expert_model_parallel_size < 1:\n            raise ValueError(\"`expert_model_parallel_size` must be >= 1.\")\n        if self.context_parallel_size < 1:\n            raise ValueError(\"`context_parallel_size` must be >= 1.\")\n        if self.virtual_pipeline_model_parallel_size is not None and self.virtual_pipeline_model_parallel_size < 1:\n            raise ValueError(\"`virtual_pipeline_model_parallel_size` must be >= 1 when set.\")\n        if self.sequence_parallel and self.tensor_model_parallel_size <= 1:\n            raise ValueError(\"`sequence_parallel` requires `tensor_model_parallel_size` > 1.\")\n        if self.recompute_granularity is not None and self.recompute_granularity not in (\"full\", \"selective\"):\n            raise ValueError(\"`recompute_granularity` must be 'full' or 'selective'.\")\n        if self.recompute_method is not None and self.recompute_method not in (\"uniform\", \"block\"):\n            raise ValueError(\"`recompute_method` must be 'uniform' or 'block'.\")\n        if self.recompute_num_layers is not None and self.recompute_num_layers < 1:\n            raise ValueError(\"`recompute_num_layers` must be >= 1 when set.\")\n        if self.moe_token_dispatcher_type is not None and self.moe_token_dispatcher_type not in (\n            \"allgather\",\n            \"alltoall\",\n            \"flex\",","sourceCodeStart":138,"sourceCodeEnd":174,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/megatron_bridge_args.py#L138-L174","documentation":"Pipeline parallelism splits transformer layers across stages; at least one stage per rank group is required, so pipeline_model_parallel_size < 1 is invalid. MegatronBridgeArgs.__post_init__ (src/llamafactory/hparams/megatron_bridge_args.py:156) enforces >= 1.","triggerScenarios":"A Megatron Bridge config with pipeline_model_parallel_size: 0 or a negative number (including values injected via extra_config overrides that bypass the YAML default).","commonSituations":"Editing a multi-node Megatron config and zeroing PP to 'turn it off' (should stay 1), or variable substitution producing 0/empty in generated configs.","solutions":["Set pipeline_model_parallel_size: 1 for no pipeline parallelism.","For real pipeline parallelism, verify world_size = TP * PP * CP * DP and that the model has enough layers to divide.","Audit extra_config JSON overrides, since they can also feed invalid values into these fields."],"exampleFix":"# before (yaml)\npipeline_model_parallel_size: 0\n\n# after (yaml)\npipeline_model_parallel_size: 1","handlingStrategy":"validation","validationCode":"def check_pp_size(pp: int) -> None:\n    if pp < 1:\n        raise ValueError(\"pipeline_model_parallel_size must be >= 1; use 1 to disable PP\")","typeGuard":"def is_valid_parallel_size(v) -> bool:\n    return isinstance(v, int) and not isinstance(v, bool) and v >= 1","tryCatchPattern":null,"preventionTips":["Keep all parallel-size fields >= 1 in generated configs; default missing fields to 1 rather than 0.","Remember total GPUs must equal TP * PP * CP * DP for Megatron runs."],"tags":["config","megatron","parallelism","distributed"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}