hiyouga/LlamaFactory · error · ValueError

`recompute_granularity` must be 'full' or 'selective'.

Error message

`recompute_granularity` must be 'full' or 'selective'.

What it means

Raised in MegatronBridgeArguments.__post_init__ (megatron_bridge_args.py:166) when recompute_granularity is set to a string other than 'full' or 'selective'. These are the only two activation-recompute granularities Megatron-core supports: 'full' recomputes every layer, 'selective' recomputes only attention/MLP cores. The validation is an exact, case-sensitive string match.

Source

Thrown at src/llamafactory/hparams/megatron_bridge_args.py:166

            )
        },
    )

    def __post_init__(self) -> None:
        if self.tensor_model_parallel_size < 1:
            raise ValueError("`tensor_model_parallel_size` must be >= 1.")
        if self.pipeline_model_parallel_size < 1:
            raise ValueError("`pipeline_model_parallel_size` must be >= 1.")
        if self.expert_model_parallel_size < 1:
            raise ValueError("`expert_model_parallel_size` must be >= 1.")
        if self.context_parallel_size < 1:
            raise ValueError("`context_parallel_size` must be >= 1.")
        if self.virtual_pipeline_model_parallel_size is not None and self.virtual_pipeline_model_parallel_size < 1:
            raise ValueError("`virtual_pipeline_model_parallel_size` must be >= 1 when set.")
        if self.sequence_parallel and self.tensor_model_parallel_size <= 1:
            raise ValueError("`sequence_parallel` requires `tensor_model_parallel_size` > 1.")
        if self.recompute_granularity is not None and self.recompute_granularity not in ("full", "selective"):
            raise ValueError("`recompute_granularity` must be 'full' or 'selective'.")
        if self.recompute_method is not None and self.recompute_method not in ("uniform", "block"):
            raise ValueError("`recompute_method` must be 'uniform' or 'block'.")
        if self.recompute_num_layers is not None and self.recompute_num_layers < 1:
            raise ValueError("`recompute_num_layers` must be >= 1 when set.")
        if self.moe_token_dispatcher_type is not None and self.moe_token_dispatcher_type not in (
            "allgather",
            "alltoall",
            "flex",
        ):
            raise ValueError("`moe_token_dispatcher_type` must be 'allgather', 'alltoall', or 'flex'.")

        if isinstance(self.extra_config, str):
            config_str = self.extra_config.strip()
            if config_str.startswith("{"):
                self.extra_config = _convert_str_dict(json.loads(config_str))
            else:
                self.extra_config = config_str

View on GitHub (pinned to f28afaf635)

Solutions

1. Use exactly 'full' or 'selective' (lowercase) for recompute_granularity
2. If you meant per-layer counting, keep recompute_granularity plus recompute_method/recompute_num_layers with valid values
3. Remove the key to accept the default behavior

Example fix

# before
recompute_granularity: Full

# after
recompute_granularity: full

if cfg.get('recompute_granularity') not in (None, 'full', 'selective'):
    raise SystemExit(f"bad recompute_granularity: {cfg['recompute_granularity']!r}")

def is_valid_granularity(v: str | None) -> bool:
    return v in (None, 'full', 'selective')

try:
    args = MegatronBridgeArguments(**cfg)
except ValueError as e:
    if 'recompute_granularity' in str(e):
        cfg['recompute_granularity'] = 'selective'  # or 'full'
        args = MegatronBridgeArguments(**cfg)
    else:
        raise

When it happens

Trigger: Passing recompute_granularity: Full (capitalized), 'FULL', 'layer', or a typo such as 'selectiev'; passing a value accepted by a different Megatron fork but not this whitelist.

Common situations: Copy-pasting from Megatron-LM docs or NVIDIA scripts that use different casing; hand-typing the value; configs migrated between LlamaFactory versions where accepted vocabularies drifted.

Related errors

• `recompute_method` must be 'uniform' or 'block'.
• `recompute_num_layers` must be >= 1 when set.
• `virtual_pipeline_model_parallel_size` must be >= 1 when set
• `sequence_parallel` requires `tensor_model_parallel_size` >
• `moe_token_dispatcher_type` must be 'allgather', 'alltoall',

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/c58cca823f0b5e5c. Report an issue: GitHub.