{"record":{"id":"3c45cd85be034201","repo":"Unity-Technologies/ml-agents","slug":"you-are-calling-step-even-though-this-environm","errorCode":null,"errorMessage":"You are calling 'step()' even though this environment has already returned `terminated` or `truncated` as True. You must always call 'reset()' once you receive `terminated` or `truncated` as True.","messagePattern":"You are calling 'step\\(\\)' even though this environment has already returned `terminated` or `truncated` as True\\. You must always call 'reset\\(\\)' once you receive `terminated` or `truncated` as True\\.","errorType":"exception","errorClass":"UnityGymException","httpStatus":null,"severity":"error","filePath":"ml-agents-envs/mlagents_envs/envs/unity_gym_env.py","lineNumber":188,"sourceCode":"        res: GymStepResult = self._single_step(decision_step)\n        return res[0], res[4]\n\n    def step(self, action: Any) -> GymStepResult:\n        \"\"\"Run one timestep of the environment's dynamics. When end of\n        episode is reached, you are responsible for calling `reset()`\n        to reset this environment's state.\n        Accepts an action and returns a tuple (observation, reward, terminated, truncated, info).\n        Args:\n            action (object/list): an action provided by the environment\n        Returns:\n            observation (object/list): agent's observation of the current environment\n            reward (float/list) : amount of reward returned after previous action\n            terminated (boolean/list): whether the episode has ended by termination.\n            truncated (boolean/list): whether the episode has ended by truncation.\n            info (dict): contains auxiliary diagnostic information.\n        \"\"\"\n        if self.game_over:\n            raise UnityGymException(\n                \"You are calling 'step()' even though this environment has already \"\n                \"returned `terminated` or `truncated` as True. You must always call 'reset()' once you \"\n                \"receive `terminated` or `truncated` as True.\"\n            )\n        if self._flattener is not None:\n            # Translate action into list\n            action = self._flattener.lookup_action(action)\n\n        action = np.array(action).reshape((1, self.action_size))\n\n        action_tuple = ActionTuple()\n        if self.group_spec.action_spec.is_continuous():\n            action_tuple.add_continuous(action)\n        else:\n            action_tuple.add_discrete(action)\n        self._env.set_actions(self.name, action_tuple)\n\n        self._env.step()","sourceCodeStart":170,"sourceCodeEnd":206,"githubUrl":"https://github.com/Unity-Technologies/ml-agents/blob/3ecb446f75d1e7400eb404c562dc005d3164cffc/ml-agents-envs/mlagents_envs/envs/unity_gym_env.py#L170-L206","documentation":"UnityGymException raised by UnityGymEnv.step() when self.game_over is True, meaning a previous step already reported terminated/truncated=True. Following the gym API, once an episode ends you must call reset() before stepping again.","triggerScenarios":"Calling env.step(action) after a step (or initial reset with done) returned terminated or truncated equal to True without calling env.reset() in between.","commonSituations":"Hand-written RL loops that ignore the done flags; stepping after the Agent reached a goal (terminated) or max-steps (truncated); custom training scripts not using stable-baselines3 or other gym loop abstractions.","solutions":["Call env.reset() immediately after receiving terminated or truncated=True before any further step().","Track the done flags in your loop and branch to reset instead of stepping.","Use a standard gym wrapper/loop (e.g. stable-baselines3 or gymnasium's Agent-Eval loop) that handles resets automatically.","Note: with _allow_multiple_obs/agent-count constraints, game_over may also be set internally — always honor the returned flags."],"exampleFix":"// before\nobs, reward, terminated, truncated, info = env.step(action)\nobs, reward, terminated, truncated, info = env.step(action)  # raises if done\n// after\nobs, reward, terminated, truncated, info = env.step(action)\nif terminated or truncated:\n    obs, info = env.reset()","handlingStrategy":"try-catch","validationCode":"if getattr(env, 'game_over', False):\n    env.reset()","typeGuard":"def can_step(env) -> bool:\n    return not getattr(env, 'game_over', False)","tryCatchPattern":"try:\n    obs, reward, terminated, truncated, info = env.step(action)\nexcept UnityGymException:\n    obs, info = env.reset()\n    obs, reward, terminated, truncated, info = env.step(action)","preventionTips":["Always reset() when terminated or truncated is True","Write the standard gym loop pattern (step -> check done -> reset)","Prefer off-the-shelf loop utilities (gymnasium eval loop, SB3) over hand-rolled loops"],"tags":["gym","episode-lifecycle","api-misuse"],"backgroundTag":"step-after-episode-done","analyzedSha":"3ecb446f75d1e7400eb404c562dc005d3164cffc","analyzedAt":"2026-09-02T16:33:12.832Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T21:17:11.164Z"}