huggingface/transformers · error · ValueError

{type(model).__name__} does not have a FSDP2 plan declared.

Error message

{type(model).__name__} does not have a FSDP2 plan declared. Set `base_model_fsdp_plan` on the config and `_fsdp_plan` on the head class.

What it means

When applying FSDP2 (torch's fully_shard), transformers requires each model class to declare which modules to wrap via an _fsdp_plan (derived from base_model_fsdp_plan on the config and _fsdp_plan on the head class). apply_fully_sharded_data_parallelism reads that plan; if the model exposes none, it refuses to guess a sharding strategy and raises. Models must opt in by declaring a plan.

Source

Thrown at src/transformers/distributed/fsdp.py:198

    if invalid_strategies:
        logger.warning(f"The following FSDP entries have unknown strategies: {invalid_strategies}")
    if unused_rules:
        logger.warning(f"The following FSDP rules were not applied to any module: {unused_rules}")


def apply_fully_sharded_data_parallelism(
    model: nn.Module, fsdp_mesh: torch.distributed.device_mesh.DeviceMesh
) -> nn.Module:
    """
    Apply FSDP2 (fully_shard) to a model.

    Torch availability, distributed initialization and the version requirement
    are asserted upstream by `initialize_fully_sharded_data_parallelism`.
    """
    fsdp_plan = dict(getattr(model, "_fsdp_plan", None) or {})
    if not fsdp_plan:
        raise ValueError(
            f"{type(model).__name__} does not have a FSDP2 plan declared. Set "
            "`base_model_fsdp_plan` on the config and `_fsdp_plan` on the head class."
        )

    distributed_config = getattr(model.config, "distributed_config", None)
    fsdp_policy_kwargs = _get_fsdp_policy_kwargs(distributed_config)

    adapted_fsdp_plan = _resolve_tied_embed_lm_head_plan(fsdp_plan, model)
    reshard_targets, no_reshard_targets = expand_fsdp_plan(model, adapted_fsdp_plan)

    for module_name, module in reshard_targets:
        fully_shard(module, mesh=fsdp_mesh, reshard_after_forward=True, **fsdp_policy_kwargs)
        logger.debug(f"Applied fully_shard to {module_name} (reshard=True)")

    # Optimization: when the keep buffer is exactly the (final_norm, lm_head/embed)
    # tail pair, bundle them into one fully_shard so that we dont need to do all-gather during backward pass.
    if is_norm_and_head_pair(no_reshard_targets, model):
        names, modules = [], []

View on GitHub (pinned to a597f97485)

Solutions

  1. Use a model that declares an FSDP2 plan (check its config for base_model_fsdp_plan / the class for _fsdp_plan).
  2. For your own model, declare the plan: set base_model_fsdp_plan on the config class and _fsdp_plan on the head class mapping module patterns to shard groups.
  3. If the model cannot be migrated, fall back to torch's native FSDP1 wrapping or plain DDP instead of this helper.

Example fix

# before
cfg = DistributedConfig(fsdp_size=8)
# model has no plan -> apply_fully_sharded_data_parallelism raises

# after: declare a plan on the model
class MyModelConfig(PretrainedConfig):
    base_model_fsdp_plan = {"language_model.layers.*": "full_shard"}

class MyModel(PreTrainedModel):
    _fsdp_plan = {"language_model.layers.*": "full_shard"}
# now fsdp_size=8 setup proceeds
Defensive patterns

Strategy: validation

Validate before calling

fsdp_plan = getattr(type(model), "_fsdp_plan", None) or getattr(model, "_fsdp_plan", None)
if distributed_config.fsdp_size and not fsdp_plan:
    raise ValueError(f"{type(model).__name__} lacks an FSDP2 plan; pick a supported model or declare one")

Type guard

def has_fsdp_plan(model) -> bool:
    return bool(getattr(model, "_fsdp_plan", None))

Try / catch

try:
    model = apply_fully_sharded_data_parallelism(model, mesh)
except ValueError as e:
    if "does not have a FSDP2 plan" in str(e):
        # fall back to DDP or a plan-declaring model
        model = model.to(rank)
    else:
        raise

Prevention

When it happens

Trigger: Enabling fsdp_size>1 in DistributedConfig and calling the FSDP2 setup path on a model whose architecture has no base_model_fsdp_plan in its config; applying apply_fully_sharded_data_parallelism directly to a custom/naive nn.Module; a newer model integration missing the plan declaration.

Common situations: FSDP training of a model family that has not yet declared an FSDP2 plan; custom model heads built outside the standard PreTrainedModel conventions; version skew where the plan mechanism is newer than the model file.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/9bea356b9b041ac5. Report an issue: GitHub.