Unity-Technologies/ml-agents · error · UnityGymException

You are calling 'step()' even though this environment has al

Error message

You are calling 'step()' even though this environment has already returned `terminated` or `truncated` as True. You must always call 'reset()' once you receive `terminated` or `truncated` as True.

What it means

UnityGymException raised by UnityGymEnv.step() when self.game_over is True, meaning a previous step already reported terminated/truncated=True. Following the gym API, once an episode ends you must call reset() before stepping again.

Source

Thrown at ml-agents-envs/mlagents_envs/envs/unity_gym_env.py:188

        res: GymStepResult = self._single_step(decision_step)
        return res[0], res[4]

    def step(self, action: Any) -> GymStepResult:
        """Run one timestep of the environment's dynamics. When end of
        episode is reached, you are responsible for calling `reset()`
        to reset this environment's state.
        Accepts an action and returns a tuple (observation, reward, terminated, truncated, info).
        Args:
            action (object/list): an action provided by the environment
        Returns:
            observation (object/list): agent's observation of the current environment
            reward (float/list) : amount of reward returned after previous action
            terminated (boolean/list): whether the episode has ended by termination.
            truncated (boolean/list): whether the episode has ended by truncation.
            info (dict): contains auxiliary diagnostic information.
        """
        if self.game_over:
            raise UnityGymException(
                "You are calling 'step()' even though this environment has already "
                "returned `terminated` or `truncated` as True. You must always call 'reset()' once you "
                "receive `terminated` or `truncated` as True."
            )
        if self._flattener is not None:
            # Translate action into list
            action = self._flattener.lookup_action(action)

        action = np.array(action).reshape((1, self.action_size))

        action_tuple = ActionTuple()
        if self.group_spec.action_spec.is_continuous():
            action_tuple.add_continuous(action)
        else:
            action_tuple.add_discrete(action)
        self._env.set_actions(self.name, action_tuple)

        self._env.step()

View on GitHub (pinned to 3ecb446f75)

Solutions

Call env.reset() immediately after receiving terminated or truncated=True before any further step().
Track the done flags in your loop and branch to reset instead of stepping.
Use a standard gym wrapper/loop (e.g. stable-baselines3 or gymnasium's Agent-Eval loop) that handles resets automatically.
Note: with _allow_multiple_obs/agent-count constraints, game_over may also be set internally — always honor the returned flags.

Example fix

// before
obs, reward, terminated, truncated, info = env.step(action)
obs, reward, terminated, truncated, info = env.step(action)  # raises if done
// after
obs, reward, terminated, truncated, info = env.step(action)
if terminated or truncated:
    obs, info = env.reset()

Defensive patterns

Strategy: try-catch

Validate before calling

if getattr(env, 'game_over', False):
    env.reset()

Type guard

def can_step(env) -> bool:
    return not getattr(env, 'game_over', False)

Try / catch

try:
    obs, reward, terminated, truncated, info = env.step(action)
except UnityGymException:
    obs, info = env.reset()
    obs, reward, terminated, truncated, info = env.step(action)

Prevention

Always reset() when terminated or truncated is True
Write the standard gym loop pattern (step -> check done -> reset)
Prefer off-the-shelf loop utilities (gymnasium eval loop, SB3) over hand-rolled loops

When it happens

Trigger: Calling env.step(action) after a step (or initial reset with done) returned terminated or truncated equal to True without calling env.reset() in between.

Common situations: Hand-written RL loops that ignore the done flags; stepping after the Agent reached a goal (terminated) or max-steps (truncated); custom training scripts not using stable-baselines3 or other gym loop abstractions.

Related errors

AI-assisted analysis of Unity-Technologies/ml-agents@3ecb446f75 (2026-09-02). Data as JSON: /api/errors/3c45cd85be034201. Report an issue: GitHub.