{"record":{"id":"6c7b20adbcbaba1c","repo":"Lightning-AI/pytorch-lightning","slug":"automatic-gradient-accumulation-is-not-supported-f","errorCode":null,"errorMessage":"Automatic gradient accumulation is not supported for manual optimization. Remove `Trainer(accumulate_grad_batches={trainer.accumulate_grad_batches})` or switch to automatic optimization.","messagePattern":"Automatic gradient accumulation is not supported for manual optimization\\. Remove `Trainer\\(accumulate_grad_batches=(.+?)\\)` or switch to automatic optimization\\.","errorType":"validation","errorClass":"MisconfigurationException","httpStatus":null,"severity":"error","filePath":"src/lightning/pytorch/trainer/configuration_validator.py","lineNumber":129,"sourceCode":"            raise NotImplementedError(\n                f\"Support for `{epoch_end_name}` has been removed in v2.0.0. `{type(model).__name__}` implements this\"\n                f\" method. You can use the `on_{epoch_end_name}` hook instead. To access outputs, save them in-memory\"\n                \" as instance attributes.\"\n                \" You can find migration examples in https://github.com/Lightning-AI/pytorch-lightning/pull/16520.\"\n            )\n\n\ndef __verify_manual_optimization_support(trainer: \"pl.Trainer\", model: \"pl.LightningModule\") -> None:\n    if model.automatic_optimization:\n        return\n    if trainer.gradient_clip_val is not None and trainer.gradient_clip_val > 0:\n        raise MisconfigurationException(\n            \"Automatic gradient clipping is not supported for manual optimization.\"\n            f\" Remove `Trainer(gradient_clip_val={trainer.gradient_clip_val})`\"\n            \" or switch to automatic optimization.\"\n        )\n    if trainer.accumulate_grad_batches != 1:\n        raise MisconfigurationException(\n            \"Automatic gradient accumulation is not supported for manual optimization.\"\n            f\" Remove `Trainer(accumulate_grad_batches={trainer.accumulate_grad_batches})`\"\n            \" or switch to automatic optimization.\"\n        )\n\n\ndef __warn_dataloader_iter_limitations(model: \"pl.LightningModule\") -> None:\n    \"\"\"Check if `dataloader_iter is enabled`.\"\"\"\n    if any(\n        is_param_in_hook_signature(step_fn, \"dataloader_iter\", explicit=True)\n        for step_fn in (model.training_step, model.validation_step, model.predict_step, model.test_step)\n        if step_fn is not None\n    ):\n        rank_zero_warn(\n            \"You are using the `dataloader_iter` step flavor. If you consume the iterator more than once per step, the\"\n            \" `batch_idx` argument in any hook that takes it will not match with the batch index of the last batch\"\n            \" consumed. This might have unforeseen effects on callbacks or code that expects to get the correct index.\"\n            \" This will also not work well with gradient accumulation. This feature is very experimental and subject to\"","sourceCodeStart":111,"sourceCodeEnd":147,"githubUrl":"https://github.com/Lightning-AI/pytorch-lightning/blob/9fed5c27d2a62ff0efd6c3573599921d6ff67c14/src/lightning/pytorch/trainer/configuration_validator.py#L111-L147","documentation":"Gradient accumulation via `Trainer(accumulate_grad_batches=k)` only works with automatic optimization; Lightning rejects any value != 1 when `automatic_optimization=False` because it cannot interleave accumulation with your manual optimizer stepping.","triggerScenarios":"`LightningModule.automatic_optimization = False` plus `Trainer(accumulate_grad_batches=4)` (or scheduler-dict accumulation settings).","commonSituations":"Memory-saving accumulation configs reused with GAN/RL manual-optimization code; accumulation set via a shared config file used by both types of modules.","solutions":["Set `accumulate_grad_batches=1` (remove it) for the manual-optimization model.","Implement accumulation yourself: only call `optimizer.step()`/`zero_grad()` every N batches using `self.global_step` or a counter.","Switch to automatic optimization to keep Trainer-level accumulation."],"exampleFix":"# before\ntrainer = Trainer(accumulate_grad_batches=8)  # manual opt model\n# after\ntrainer = Trainer()\n# in training_step, step every 8 batches:\n# if (self._step + 1) % 8 == 0: opt.step(); opt.zero_grad()","handlingStrategy":"validation","validationCode":"def validate_accumulation(model, trainer_kwargs):\n    if getattr(model, 'automatic_optimization', True) is False and trainer_kwargs.get('accumulate_grad_batches', 1) != 1:\n        raise ValueError('Set accumulate_grad_batches=1 and accumulate manually in training_step')\n    return trainer_kwargs","typeGuard":"def needs_manual_accumulation(model) -> bool:\n    return model.automatic_optimization is False","tryCatchPattern":"except MisconfigurationException as e: if 'gradient accumulation' in str(e): rebuild Trainer with accumulate_grad_batches=1 and add custom stepping","preventionTips":["Separate config presets for manual vs automatic optimization models.","Unit-test trainer construction for each model/config pair."],"tags":["pytorch-lightning","manual-optimization","gradient-accumulation","configuration"],"backgroundTag":"incompatible-trainer-options","analyzedSha":"9fed5c27d2a62ff0efd6c3573599921d6ff67c14","analyzedAt":"2026-08-28T11:52:41.083Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}