{"record":{"id":"41d75adfc3d48fe3","repo":"Unity-Technologies/ml-agents","slug":"inf-found","errorCode":null,"errorMessage":"Inf found","messagePattern":"Inf found","errorType":"exception","errorClass":"UnityTrainerException","httpStatus":null,"severity":"critical","filePath":"ml-agents/mlagents/trainers/sac/optimizer_torch.py","lineNumber":366,"sourceCode":"                with torch.no_grad():\n                    v_backup = min_policy_qs[name] - torch.mean(\n                        branched_ent_bonus, axis=0\n                    )\n                    # Add continuous entropy bonus to minimum Q\n                    if self._action_spec.continuous_size > 0:\n                        v_backup += torch.sum(\n                            _cont_ent_coef * log_probs.continuous_tensor,\n                            dim=1,\n                            keepdim=True,\n                        )\n                value_loss = 0.5 * ModelUtils.masked_mean(\n                    torch.nn.functional.mse_loss(values[name], v_backup.squeeze()),\n                    loss_masks,\n                )\n                value_losses.append(value_loss)\n        value_loss = torch.mean(torch.stack(value_losses))\n        if torch.isinf(value_loss).any() or torch.isnan(value_loss).any():\n            raise UnityTrainerException(\"Inf found\")\n        return value_loss\n\n    def sac_policy_loss(\n        self,\n        log_probs: ActionLogProbs,\n        q1p_outs: Dict[str, torch.Tensor],\n        loss_masks: torch.Tensor,\n    ) -> torch.Tensor:\n        _cont_ent_coef, _disc_ent_coef = (\n            self._log_ent_coef.continuous,\n            self._log_ent_coef.discrete,\n        )\n        _cont_ent_coef = _cont_ent_coef.exp()\n        _disc_ent_coef = _disc_ent_coef.exp()\n\n        mean_q1 = torch.mean(torch.stack(list(q1p_outs.values())), axis=0)\n        batch_policy_loss = 0\n        if self._action_spec.discrete_size > 0:","sourceCodeStart":348,"sourceCodeEnd":384,"githubUrl":"https://github.com/Unity-Technologies/ml-agents/blob/3ecb446f75d1e7400eb404c562dc005d3164cffc/ml-agents/mlagents/trainers/sac/optimizer_torch.py#L348-L384","documentation":"UnityTrainerException raised in sac_value_loss when the aggregated critic value loss contains Inf or NaN. After computing MSE losses between value estimates and backup targets for each reward signal, the optimizer checks torch.isinf/torch.isnan on the summed value loss and aborts the update to prevent corrupting network weights.","triggerScenarios":"During SAC update() when value targets explode — e.g. extremely large rewards/reward signal strengths, discount factor near 1 with non-terminal bootstrap, buffer of bad data, or the Q/value networks already containing NaN from a previous diverged update.","commonSituations":"Reward signals with huge magnitudes (e.g. unscaled extrinsic + curiosity); learning rate too high causing divergence mid-training; reward signal strength settings producing backup values in the thousands+.","solutions":["Lower the learning rate and/or add/keep gradient clipping in SAC hyperparameters to stop divergence.","Scale down reward_signal strengths so value targets stay in a reasonable range.","Check reward function output for Inf/NaN before training; sanitize the environment's rewards.","Restart from an earlier checkpoint (via --init-file) taken before the value loss blew up."],"exampleFix":"# before\nrewards:\n  extrinsic:\n    strength: 100.0\n# after\nrewards:\n  extrinsic:\n    strength: 1.0","handlingStrategy":"validation","validationCode":"import numpy as np\nassert np.isfinite(env_reward).all(), \"Environment emits non-finite rewards\"\nassert all(0 < s <= 10 for s in reward_signal_strengths), \"Reward strengths too large\"","typeGuard":"import torch\ndef value_loss_is_finite(loss: torch.Tensor) -> bool:\n    return bool(torch.isfinite(loss).all())","tryCatchPattern":"from mlagents.trainers.exception import UnityTrainerException\ntry:\n    optimizer.update(batch, num_sequences)\nexcept UnityTrainerException as e:\n    if \"Inf found\" in str(e):\n        logger.error(\"SAC value loss diverged; loading last stable checkpoint\")\n        policy.load(last_good_checkpoint)","preventionTips":["Keep reward signal strengths modest (extrinsic ~1.0)","Reject or clip non-finite rewards at the environment interface","Use a conservative SAC learning rate and monitor value loss on TensorBoard","Snapshot checkpoints frequently for rollback after divergence"],"tags":["ml-agents","sac","nan","inf","training-divergence"],"backgroundTag":"loss-diverged-to-inf","analyzedSha":"3ecb446f75d1e7400eb404c562dc005d3164cffc","analyzedAt":"2026-09-02T16:33:12.832Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T21:17:11.164Z"}