{"record":{"id":"5448abd7a54f13c4","repo":"hiyouga/LlamaFactory","slug":"context-parallel-size-must-be-1","errorCode":null,"errorMessage":"`context_parallel_size` must be >= 1.","messagePattern":"`context_parallel_size` must be >= 1\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/megatron_bridge_args.py","lineNumber":160,"sourceCode":"    extra_config: Optional[str] = field(\n        default=None,\n        metadata={\n            \"help\": (\n                \"Optional JSON string or path to a JSON file with extra Megatron Bridge model/training overrides. \"\n                \"Dot-paths are supported (e.g. train.train_iters or checkpoint.save_interval).\"\n            )\n        },\n    )\n\n    def __post_init__(self) -> None:\n        if self.tensor_model_parallel_size < 1:\n            raise ValueError(\"`tensor_model_parallel_size` must be >= 1.\")\n        if self.pipeline_model_parallel_size < 1:\n            raise ValueError(\"`pipeline_model_parallel_size` must be >= 1.\")\n        if self.expert_model_parallel_size < 1:\n            raise ValueError(\"`expert_model_parallel_size` must be >= 1.\")\n        if self.context_parallel_size < 1:\n            raise ValueError(\"`context_parallel_size` must be >= 1.\")\n        if self.virtual_pipeline_model_parallel_size is not None and self.virtual_pipeline_model_parallel_size < 1:\n            raise ValueError(\"`virtual_pipeline_model_parallel_size` must be >= 1 when set.\")\n        if self.sequence_parallel and self.tensor_model_parallel_size <= 1:\n            raise ValueError(\"`sequence_parallel` requires `tensor_model_parallel_size` > 1.\")\n        if self.recompute_granularity is not None and self.recompute_granularity not in (\"full\", \"selective\"):\n            raise ValueError(\"`recompute_granularity` must be 'full' or 'selective'.\")\n        if self.recompute_method is not None and self.recompute_method not in (\"uniform\", \"block\"):\n            raise ValueError(\"`recompute_method` must be 'uniform' or 'block'.\")\n        if self.recompute_num_layers is not None and self.recompute_num_layers < 1:\n            raise ValueError(\"`recompute_num_layers` must be >= 1 when set.\")\n        if self.moe_token_dispatcher_type is not None and self.moe_token_dispatcher_type not in (\n            \"allgather\",\n            \"alltoall\",\n            \"flex\",\n        ):\n            raise ValueError(\"`moe_token_dispatcher_type` must be 'allgather', 'alltoall', or 'flex'.\")\n\n        if isinstance(self.extra_config, str):","sourceCodeStart":142,"sourceCodeEnd":178,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/megatron_bridge_args.py#L142-L178","documentation":"Context parallelism shards the sequence dimension across ranks for long-context training; fewer than one context-parallel rank is invalid. MegatronBridgeArgs.__post_init__ (src/llamafactory/hparams/megatron_bridge_args.py:160) requires context_parallel_size >= 1.","triggerScenarios":"A Megatron Bridge config with context_parallel_size: 0 or negative — often hand-set to disable CP, or produced by a config template where the field is conditionally populated.","commonSituations":"Long-context (32k+) training configs where users toggle CP on/off by editing numbers, and scripts that emit 0 for 'off'.","solutions":["Set context_parallel_size: 1 to disable context parallelism.","When enabling CP for long sequences, ensure sequence length and world size are divisible appropriately and pair with recompute settings.","Remove the key entirely to fall back to the default of 1 instead of writing 0."],"exampleFix":"# before (yaml)\ncontext_parallel_size: 0\n\n# after (yaml)\ncontext_parallel_size: 1","handlingStrategy":"validation","validationCode":"def check_cp_size(cp: int) -> None:\n    if cp < 1:\n        raise ValueError(\"context_parallel_size must be >= 1; use 1 to disable CP\")","typeGuard":"def is_valid_parallel_size(v) -> bool:\n    return isinstance(v, int) and not isinstance(v, bool) and v >= 1","tryCatchPattern":null,"preventionTips":["Disable context parallelism by setting 1 (or omitting the key), not 0.","When CP > 1, pair it with activation recomputation settings for long-context memory savings."],"tags":["config","megatron","context-parallelism","long-context"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}