Lightning-AI/pytorch-lightning · error · RuntimeError

The hybrid sharding strategy requires you to pass at least o

Error message

The hybrid sharding strategy requires you to pass at least one of the parameters: `auto_wrap_policy`, `process_group` tuple, or `device_mesh`.

What it means

This error is raised by Lightning Fabric's FSDP strategy when the user selects a hybrid sharding variant ('HYBRID_SHARD' or '_HYBRID_SHARD_ZERO2') but does not supply any of the three parameters that define the intra-node group boundaries: auto_wrap_policy, a process_group tuple, or a device_mesh. Hybrid sharding needs to know which ranks form the inner shard group versus the outer replicated group; without one of these, the sharding topology is undefined and Lightning refuses to construct the strategy.

Source

Thrown at src/lightning/fabric/strategies/fsdp.py:870

def _init_sharding_strategy(sharding_strategy: "_SHARDING_STRATEGY", kwargs: dict) -> "ShardingStrategy":
    from torch.distributed.fsdp import ShardingStrategy

    if kwargs.get("process_group") is not None and kwargs.get("device_mesh") is not None:
        raise ValueError(
            "The arguments `FSDPStrategy(process_group=..., device_mesh=...)` are mutually exclusive."
            "Pass only one of them."
        )

    strategy = ShardingStrategy[sharding_strategy.upper()] if isinstance(sharding_strategy, str) else sharding_strategy
    if (
        "HYBRID" in strategy.name
        and kwargs.get("auto_wrap_policy") is None
        and kwargs.get("process_group") is None
        and kwargs.get("device_mesh") is None
    ):
        raise RuntimeError(
            "The hybrid sharding strategy requires you to pass at least one of the parameters: `auto_wrap_policy`,"
            " `process_group` tuple, or `device_mesh`."
        )
    return strategy


def _optimizer_has_flat_params(optimizer: Optimizer) -> bool:
    return any(
        getattr(param, "_fsdp_flattened", False) for group in optimizer.param_groups for param in group["params"]
    )


def _get_sharded_state_dict_context(module: Module) -> Generator[None, None, None]:
    from torch.distributed.fsdp import FullyShardedDataParallel as FSDP
    from torch.distributed.fsdp.api import ShardedOptimStateDictConfig, ShardedStateDictConfig, StateDictType

    state_dict_config = ShardedStateDictConfig(offload_to_cpu=True)
    optim_state_dict_config = ShardedOptimStateDictConfig(offload_to_cpu=True)

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Pass an auto_wrap_policy, e.g. size_based_auto_wrap_policy from torch.distributed.fsdp.wrap, to FSDPStrategy
2. Or pass a process_group tuple: process_group=(intra_node_group, inter_node_group) created with torch.distributed.new_group
3. Or pass a torch.distributed.device_mesh.DeviceMesh initialized over your world
4. If you don't need intra-node sharding with inter-node replication, use the plain 'fsdp' strategy instead

Example fix

# before
strategy = FSDPStrategy(sharding_strategy="HYBRID_SHARD")

# after
from torch.distributed.fsdp.wrap import size_based_auto_wrap_policy
strategy = FSDPStrategy(
    sharding_strategy="HYBRID_SHARD",
    auto_wrap_policy=size_based_auto_wrap_policy,
)

kwargs = {"auto_wrap_policy": None, "process_group": None, "device_mesh": None}
# or your actual FSDPStrategy kwargs
if "HYBRID" in str(sharding_strategy).upper() or "hybrid" in str(strategy_name):
    if all(v is None for v in (kwargs.get("auto_wrap_policy"), kwargs.get("process_group"), kwargs.get("device_mesh"))):
        raise ValueError("HYBRID sharding needs auto_wrap_policy, process_group, or device_mesh")

When it happens

Trigger: Calling Fabric(strategy='fsdp_hybrid_shard') or FSDPStrategy(sharding_strategy='HYBRID_SHARD') (or the _HYBRID_SHARD_ZERO2 variants) without passing auto_wrap_policy, process_group=(...), or device_mesh=... to FSDPStrategy/__init__.

Common situations: Switching an existing FSDP config string from 'fsdp' to 'fsdp_hybrid_shard' assuming it works out of the box; upgrading examples that only set sharding_strategy; copying a plain FSDP setup into a multi-node job where hybrid sharding was recommended.

Related errors

• The optimizer has references to the model's meta-device para
• Found multiple FSDP models in the given state. Saving checkp
• Found multiple FSDP models in the given state. Loading check
• `precision={precision!r})` is not supported in FSDP. `precis
• `gradient_clip_algorithm='norm'` is currently not supported

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/55552f3f43fd1881. Report an issue: GitHub.