Lightning-AI/pytorch-lightning · error · ValueError

The `ModelParallelStrategy` does not support `Fabric(..., pr

Error message

The `ModelParallelStrategy` does not support `Fabric(..., precision={self._precision_flag!r})`. Choose a different precision among: {', '.join(mp_precision_supported)}.

What it means

ModelParallelStrategy (used for large-model sharding like FSDP2/torch distributed tensor flows) only supports a restricted precision set: '32-true', 'bf16-mixed', 'bf16-true', '16-true'. Requesting e.g. '16-mixed' or '64-true' raises ValueError. The message mistakenly references Fabric but applies to Trainer.

Source

Thrown at src/lightning/pytorch/trainer/connectors/accelerator_connector.py:505

                f"Using {'16bit' if self._precision_flag == '16-mixed' else 'bfloat16'} Automatic Mixed Precision (AMP)"
            )
            device = self._accelerator_flag if self._accelerator_flag in ("cpu", "mps") else "cuda"
            return MixedPrecision(self._precision_flag, device)

        raise RuntimeError("No precision set")

    def _validate_precision_choice(self) -> None:
        """Validate the combination of choices for precision, AMP type, and accelerator."""
        if isinstance(self._precision_plugin_flag, BitsandbytesPrecision) and not isinstance(
            self.accelerator, CUDAAccelerator
        ):
            raise RuntimeError("Bitsandbytes is only supported on CUDA GPUs.")
        mp_precision_supported = ("32-true", "bf16-mixed", "bf16-true", "16-true")
        if (
            isinstance(self._strategy_flag, ModelParallelStrategy)
            and self._precision_flag not in mp_precision_supported
        ):
            raise ValueError(
                f"The `ModelParallelStrategy` does not support `Fabric(..., precision={self._precision_flag!r})`."
                f" Choose a different precision among: {', '.join(mp_precision_supported)}."
            )

    def _lazy_init_strategy(self) -> None:
        """Lazily set missing attributes on the previously instantiated strategy."""
        self.strategy.accelerator = self.accelerator
        if self.precision_plugin:
            self.strategy.precision_plugin = self.precision_plugin
        if self.checkpoint_io:
            self.strategy.checkpoint_io = self.checkpoint_io
        if hasattr(self.strategy, "cluster_environment"):
            if self.strategy.cluster_environment is None:
                self.strategy.cluster_environment = self.cluster_environment
            self.cluster_environment = self.strategy.cluster_environment
        if hasattr(self.strategy, "parallel_devices"):
            if self.strategy.parallel_devices:
                self._parallel_devices = self.strategy.parallel_devices

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Switch precision to a supported value, typically 'bf16-mixed' (recommended for model parallel)
2. If fp16 AMP is required, use a different strategy (e.g. FSDPStrategy) that supports '16-mixed'

Example fix

# before
trainer = Trainer(strategy=ModelParallelStrategy(), precision="16-mixed")
# after
trainer = Trainer(strategy=ModelParallelStrategy(), precision="bf16-mixed")

MP_SUPPORTED = ("32-true", "bf16-mixed", "bf16-true", "16-true")
if isinstance(strategy, ModelParallelStrategy) and precision not in MP_SUPPORTED:
    precision = "bf16-mixed"
trainer = Trainer(strategy=strategy, precision=precision)

def model_parallel_precision_ok(strategy, precision) -> bool:
    MP = ("32-true", "bf16-mixed", "bf16-true", "16-true")
    from lightning.pytorch.strategies import ModelParallelStrategy
    return not isinstance(strategy, ModelParallelStrategy) or precision in MP

When it happens

Trigger: Trainer(strategy=ModelParallelStrategy(), precision='16-mixed') or precision='64-true' with the model-parallel strategy selected.

Common situations: Adapting mixed-precision training scripts to model-parallel sharding; combining fp16 AMP configs with new-style model parallel strategies.

Related errors

• Passed `{type(self).__name__}(precision={precision!r})`. Pre
• `precision='bf16-mixed'` does not use a scaler, found {scale
• `precision={precision!r})` is not supported in DeepSpeed. `p
• `precision={precision!r})` is not supported in FSDP. `precis
• `precision={precision!r}` does not use a scaler, found {scal

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/940656c8b6827c4c. Report an issue: GitHub.