Unity-Technologies/ml-agents · critical · RuntimeError

Continuous NaN action detected.

Error message

Continuous NaN action detected.

What it means

RuntimeError raised by Policy.check_nan_action when the continuous portion of an action contains NaN. It sums the continuous action array and checks with np.isnan — a fast NaN detection used after every inference to prevent NaN values from corrupting the training buffer and network weights. Called from get_action in the inference path.

Source

Thrown at ml-agents/mlagents/trainers/policy/policy.py:126

    def remove_previous_action(self, agent_ids: List[GlobalAgentId]) -> None:
        for agent_id in agent_ids:
            if agent_id in self.previous_action_dict:
                self.previous_action_dict.pop(agent_id)

    def get_action(
        self, decision_requests: DecisionSteps, worker_id: int = 0
    ) -> ActionInfo:
        raise NotImplementedError

    @staticmethod
    def check_nan_action(action: Optional[ActionTuple]) -> None:
        # Fast NaN check on the action
        # See https://stackoverflow.com/questions/6736590/fast-check-for-nan-in-numpy for background.
        if action is not None:
            d = np.sum(action.continuous)
            has_nan = np.isnan(d)
            if has_nan:
                raise RuntimeError("Continuous NaN action detected.")

    @abstractmethod
    def increment_step(self, n_steps):
        pass

    @abstractmethod
    def get_current_step(self):
        pass

    @abstractmethod
    def load_weights(self, values: List[np.ndarray]) -> None:
        pass

    @abstractmethod
    def get_weights(self) -> List[np.ndarray]:
        return []

    @abstractmethod

View on GitHub (pinned to 3ecb446f75)

Solutions

1. Inspect your observations for NaN/Inf before passing them to the policy (np.isnan(obs).any()); fix the environment or normalize observations.
2. Reduce the learning rate and/or gradient clipping in your trainer settings to stop divergence producing NaN weights.
3. If it appears at startup, re-export or retrain your .pt checkpoint — the loaded weights are likely corrupted.
4. Re-run training from an earlier checkpoint taken before NaNs appeared.

Example fix

# before
action = policy.get_action(decision_steps)  # crashes with NaN continuous actions
# after
obs = decision_steps.obs
if any(np.isnan(o).any() for o in obs):
    obs = [np.nan_to_num(o) for o in obs]
action = policy.get_action(decision_steps)

import numpy as np
def observations_are_finite(decision_steps) -> bool:
    return all(np.isfinite(o).all() for o in decision_steps.obs)

def is_finite_action(action) -> bool:
    import numpy as np
    if action is None:
        return True
    return np.isfinite(np.sum(action.continuous))

try:
    action_info = policy.get_action(decision_steps, worker_id)
except RuntimeError as e:
    if "NaN" in str(e):
        logger.error("Policy produced NaN actions; restoring last good checkpoint")
        policy.load(last_good_checkpoint)

When it happens

Trigger: Any inference step where the policy's network outputs NaN in the continuous action head — usually caused by exploding/vanishing values, a learning rate that's too high, unnormalized observations (inf/NaN inputs), or a corrupted checkpoint loaded via --init-file.

Common situations: Training diverges mid-run after reward/observation magnitudes blow up; loading a partially-written or corrupt .pt checkpoint; environment sends NaN observations (sensor error) that propagate through the network.

Related errors

• Inf found
• Index out of bounds, expected a number between 0 and {Length
• Enumerator not started.
• Enumerator has reached the end already.
• Action spaces with both continuous and discrete actions are

AI-assisted analysis of Unity-Technologies/ml-agents@3ecb446f75 (2026-09-02). Data as JSON: /api/errors/8e43982af507ea03. Report an issue: GitHub.