{"record":{"id":"6c69ef53b116b76a","repo":"hiyouga/LlamaFactory","slug":"virtual-pipeline-model-parallel-size-must-be","errorCode":null,"errorMessage":"`virtual_pipeline_model_parallel_size` must be >= 1 when set.","messagePattern":"`virtual_pipeline_model_parallel_size` must be >= 1 when set\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/megatron_bridge_args.py","lineNumber":162,"sourceCode":"        metadata={\n            \"help\": (\n                \"Optional JSON string or path to a JSON file with extra Megatron Bridge model/training overrides. \"\n                \"Dot-paths are supported (e.g. train.train_iters or checkpoint.save_interval).\"\n            )\n        },\n    )\n\n    def __post_init__(self) -> None:\n        if self.tensor_model_parallel_size < 1:\n            raise ValueError(\"`tensor_model_parallel_size` must be >= 1.\")\n        if self.pipeline_model_parallel_size < 1:\n            raise ValueError(\"`pipeline_model_parallel_size` must be >= 1.\")\n        if self.expert_model_parallel_size < 1:\n            raise ValueError(\"`expert_model_parallel_size` must be >= 1.\")\n        if self.context_parallel_size < 1:\n            raise ValueError(\"`context_parallel_size` must be >= 1.\")\n        if self.virtual_pipeline_model_parallel_size is not None and self.virtual_pipeline_model_parallel_size < 1:\n            raise ValueError(\"`virtual_pipeline_model_parallel_size` must be >= 1 when set.\")\n        if self.sequence_parallel and self.tensor_model_parallel_size <= 1:\n            raise ValueError(\"`sequence_parallel` requires `tensor_model_parallel_size` > 1.\")\n        if self.recompute_granularity is not None and self.recompute_granularity not in (\"full\", \"selective\"):\n            raise ValueError(\"`recompute_granularity` must be 'full' or 'selective'.\")\n        if self.recompute_method is not None and self.recompute_method not in (\"uniform\", \"block\"):\n            raise ValueError(\"`recompute_method` must be 'uniform' or 'block'.\")\n        if self.recompute_num_layers is not None and self.recompute_num_layers < 1:\n            raise ValueError(\"`recompute_num_layers` must be >= 1 when set.\")\n        if self.moe_token_dispatcher_type is not None and self.moe_token_dispatcher_type not in (\n            \"allgather\",\n            \"alltoall\",\n            \"flex\",\n        ):\n            raise ValueError(\"`moe_token_dispatcher_type` must be 'allgather', 'alltoall', or 'flex'.\")\n\n        if isinstance(self.extra_config, str):\n            config_str = self.extra_config.strip()\n            if config_str.startswith(\"{\"):","sourceCodeStart":144,"sourceCodeEnd":180,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/megatron_bridge_args.py#L144-L180","documentation":"Raised in MegatronBridgeArguments.__post_init__ (src/llamafactory/hparams/megatron_bridge_args.py:162) when virtual_pipeline_model_parallel_size is explicitly set (not None) to a value below 1. Virtual pipeline (interleaved) parallelism splits each pipeline stage into multiple model chunks, so the count must be a positive integer. The check runs at argument-construction time, before any training starts.","triggerScenarios":"Setting virtual_pipeline_model_parallel_size: 0 (e.g. when templating a YAML config and the variable interpolates to 0) or a negative value in the megatron section of the training config. Leaving it unset (None) does not raise.","commonSituations":"Auto-generated configs that default numeric fields to 0 instead of omitting them; copying a Megatron-LM reference config into LlamaFactory's schema; users assuming 0 means 'disabled' like other flags.","solutions":["Remove virtual_pipeline_model_parallel_size from the YAML to disable it (None is the disabled state)","Set it to a positive integer (typically 2-8, and it must divide the number of transformer layers per pipeline stage)","If templating configs, emit the key only when the value is defined and > 0"],"exampleFix":"# before\nmegatron_bridge_args:\n  virtual_pipeline_model_parallel_size: 0\n\n# after\nmegatron_bridge_args:\n  virtual_pipeline_model_parallel_size: 2\n# or omit the key entirely","handlingStrategy":"validation","validationCode":"# before building args\nvpp = cfg.get('virtual_pipeline_model_parallel_size')\nif vpp is not None and vpp < 1:\n    del cfg['virtual_pipeline_model_parallel_size']  # or raise with your own message\nargs = MegatronBridgeArguments(**cfg)","typeGuard":"def valid_vpp(v: int | None) -> bool:\n    return v is None or (isinstance(v, int) and v >= 1)","tryCatchPattern":null,"preventionTips":["Treat optional numeric megatron fields as omit-means-off, never default them to 0 in templates","Add a config linter step that flags set-but-invalid parallel sizes before launch"],"tags":["megatron","parallelism","config-validation","hparams"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}