hiyouga/LlamaFactory · error · ValueError

`recompute_num_layers` must be >= 1 when set.

Error message

`recompute_num_layers` must be >= 1 when set.

What it means

Raised in MegatronBridgeArguments.__post_init__ (megatron_bridge_args.py:170) when recompute_num_layers is set (not None) to a value below 1. This field says how many transformer layers per recompute block/chunk get recomputed, so zero or negative values are nonsensical. It is only checked when the field is present; None skips validation.

Source

Thrown at src/llamafactory/hparams/megatron_bridge_args.py:170

    def __post_init__(self) -> None:
        if self.tensor_model_parallel_size < 1:
            raise ValueError("`tensor_model_parallel_size` must be >= 1.")
        if self.pipeline_model_parallel_size < 1:
            raise ValueError("`pipeline_model_parallel_size` must be >= 1.")
        if self.expert_model_parallel_size < 1:
            raise ValueError("`expert_model_parallel_size` must be >= 1.")
        if self.context_parallel_size < 1:
            raise ValueError("`context_parallel_size` must be >= 1.")
        if self.virtual_pipeline_model_parallel_size is not None and self.virtual_pipeline_model_parallel_size < 1:
            raise ValueError("`virtual_pipeline_model_parallel_size` must be >= 1 when set.")
        if self.sequence_parallel and self.tensor_model_parallel_size <= 1:
            raise ValueError("`sequence_parallel` requires `tensor_model_parallel_size` > 1.")
        if self.recompute_granularity is not None and self.recompute_granularity not in ("full", "selective"):
            raise ValueError("`recompute_granularity` must be 'full' or 'selective'.")
        if self.recompute_method is not None and self.recompute_method not in ("uniform", "block"):
            raise ValueError("`recompute_method` must be 'uniform' or 'block'.")
        if self.recompute_num_layers is not None and self.recompute_num_layers < 1:
            raise ValueError("`recompute_num_layers` must be >= 1 when set.")
        if self.moe_token_dispatcher_type is not None and self.moe_token_dispatcher_type not in (
            "allgather",
            "alltoall",
            "flex",
        ):
            raise ValueError("`moe_token_dispatcher_type` must be 'allgather', 'alltoall', or 'flex'.")

        if isinstance(self.extra_config, str):
            config_str = self.extra_config.strip()
            if config_str.startswith("{"):
                self.extra_config = _convert_str_dict(json.loads(config_str))
            else:
                self.extra_config = config_str

    def load_extra_config(self) -> dict:
        if self.extra_config is None:
            return {}
        if isinstance(self.extra_config, dict):

View on GitHub (pinned to f28afaf635)

Solutions

1. Set recompute_num_layers to >= 1 (1 is the common value)
2. Omit the key entirely if you do not want layer-count-based recomputation
3. Fix the templating layer so optional numerics are omitted rather than zeroed

Example fix

# before
recompute_num_layers: 0

# after
recompute_num_layers: 1
# or omit the key

n = cfg.get('recompute_num_layers')
if n is not None and n < 1:
    del cfg['recompute_num_layers']
args = MegatronBridgeArguments(**cfg)

def valid_layer_count(v: int | None) -> bool:
    return v is None or (isinstance(v, int) and v >= 1)

When it happens

Trigger: Templated YAML where recompute_num_layers defaults to 0; passing -1 intending 'auto'; setting 0 intending to disable recomputation (the correct way is to leave recompute_granularity unset or None).

Common situations: Config generators that emit 0 for every unset numeric field; users mixing up recompute_num_layers with a boolean-style switch; downgrading from per-layer configs that used fractional logic.

Related errors

• `recompute_granularity` must be 'full' or 'selective'.
• `recompute_method` must be 'uniform' or 'block'.
• `virtual_pipeline_model_parallel_size` must be >= 1 when set
• `sequence_parallel` requires `tensor_model_parallel_size` >
• `moe_token_dispatcher_type` must be 'allgather', 'alltoall',

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/352cd4872e004e3c. Report an issue: GitHub.