{"record":{"id":"352cd4872e004e3c","repo":"hiyouga/LlamaFactory","slug":"recompute-num-layers-must-be-1-when-set","errorCode":null,"errorMessage":"`recompute_num_layers` must be >= 1 when set.","messagePattern":"`recompute_num_layers` must be >= 1 when set\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/hparams/megatron_bridge_args.py","lineNumber":170,"sourceCode":"    def __post_init__(self) -> None:\n        if self.tensor_model_parallel_size < 1:\n            raise ValueError(\"`tensor_model_parallel_size` must be >= 1.\")\n        if self.pipeline_model_parallel_size < 1:\n            raise ValueError(\"`pipeline_model_parallel_size` must be >= 1.\")\n        if self.expert_model_parallel_size < 1:\n            raise ValueError(\"`expert_model_parallel_size` must be >= 1.\")\n        if self.context_parallel_size < 1:\n            raise ValueError(\"`context_parallel_size` must be >= 1.\")\n        if self.virtual_pipeline_model_parallel_size is not None and self.virtual_pipeline_model_parallel_size < 1:\n            raise ValueError(\"`virtual_pipeline_model_parallel_size` must be >= 1 when set.\")\n        if self.sequence_parallel and self.tensor_model_parallel_size <= 1:\n            raise ValueError(\"`sequence_parallel` requires `tensor_model_parallel_size` > 1.\")\n        if self.recompute_granularity is not None and self.recompute_granularity not in (\"full\", \"selective\"):\n            raise ValueError(\"`recompute_granularity` must be 'full' or 'selective'.\")\n        if self.recompute_method is not None and self.recompute_method not in (\"uniform\", \"block\"):\n            raise ValueError(\"`recompute_method` must be 'uniform' or 'block'.\")\n        if self.recompute_num_layers is not None and self.recompute_num_layers < 1:\n            raise ValueError(\"`recompute_num_layers` must be >= 1 when set.\")\n        if self.moe_token_dispatcher_type is not None and self.moe_token_dispatcher_type not in (\n            \"allgather\",\n            \"alltoall\",\n            \"flex\",\n        ):\n            raise ValueError(\"`moe_token_dispatcher_type` must be 'allgather', 'alltoall', or 'flex'.\")\n\n        if isinstance(self.extra_config, str):\n            config_str = self.extra_config.strip()\n            if config_str.startswith(\"{\"):\n                self.extra_config = _convert_str_dict(json.loads(config_str))\n            else:\n                self.extra_config = config_str\n\n    def load_extra_config(self) -> dict:\n        if self.extra_config is None:\n            return {}\n        if isinstance(self.extra_config, dict):","sourceCodeStart":152,"sourceCodeEnd":188,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/hparams/megatron_bridge_args.py#L152-L188","documentation":"Raised in MegatronBridgeArguments.__post_init__ (megatron_bridge_args.py:170) when recompute_num_layers is set (not None) to a value below 1. This field says how many transformer layers per recompute block/chunk get recomputed, so zero or negative values are nonsensical. It is only checked when the field is present; None skips validation.","triggerScenarios":"Templated YAML where recompute_num_layers defaults to 0; passing -1 intending 'auto'; setting 0 intending to disable recomputation (the correct way is to leave recompute_granularity unset or None).","commonSituations":"Config generators that emit 0 for every unset numeric field; users mixing up recompute_num_layers with a boolean-style switch; downgrading from per-layer configs that used fractional logic.","solutions":["Set recompute_num_layers to >= 1 (1 is the common value)","Omit the key entirely if you do not want layer-count-based recomputation","Fix the templating layer so optional numerics are omitted rather than zeroed"],"exampleFix":"# before\nrecompute_num_layers: 0\n\n# after\nrecompute_num_layers: 1\n# or omit the key","handlingStrategy":"validation","validationCode":"n = cfg.get('recompute_num_layers')\nif n is not None and n < 1:\n    del cfg['recompute_num_layers']\nargs = MegatronBridgeArguments(**cfg)","typeGuard":"def valid_layer_count(v: int | None) -> bool:\n    return v is None or (isinstance(v, int) and v >= 1)","tryCatchPattern":null,"preventionTips":["Never emit 0 for optional counts in generated configs; omit the key","Remember disabled == absent, not zero, for this field"],"tags":["megatron","activation-checkpointing","config-validation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}