Unity-Technologies/ml-agents · error · KeyError

agent_id {agent_id} is not present in the TerminalSteps

Error message

agent_id {agent_id} is not present in the TerminalSteps

What it means

TerminalSteps.__getitem__ raises KeyError when the requested agent_id is not among the agents that terminated this step. Like DecisionSteps, only ids present in the current terminal batch are valid keys in agent_id_to_index. The library raises this rather than returning a default so callers don't silently treat non-terminated agents as done.

Source

Thrown at ml-agents-envs/mlagents_envs/base_env.py:230

        """
        if self._agent_id_to_index is None:
            self._agent_id_to_index = {}
            for a_idx, a_id in enumerate(self.agent_id):
                self._agent_id_to_index[a_id] = a_idx
        return self._agent_id_to_index

    def __len__(self) -> int:
        return len(self.agent_id)

    def __getitem__(self, agent_id: AgentId) -> TerminalStep:
        """
        returns the TerminalStep for a specific agent.
        :param agent_id: The id of the agent
        :returns: obs, reward, done, agent_id and optional action mask for a
        specific agent
        """
        if agent_id not in self.agent_id_to_index:
            raise KeyError(f"agent_id {agent_id} is not present in the TerminalSteps")
        agent_index = self._agent_id_to_index[agent_id]  # type: ignore
        agent_obs = []
        for batched_obs in self.obs:
            agent_obs.append(batched_obs[agent_index])
        group_id = self.group_id[agent_index]
        return TerminalStep(
            obs=agent_obs,
            reward=self.reward[agent_index],
            interrupted=self.interrupted[agent_index],
            agent_id=agent_id,
            group_id=group_id,
            group_reward=self.group_reward[agent_index],
        )

    def __iter__(self) -> Iterator[Any]:
        yield from self.agent_id

    @staticmethod

View on GitHub (pinned to 3ecb446f75)

Solutions

  1. Guard with `if agent_id in terminal_steps:` before indexing.
  2. Only iterate terminal_steps.agent_id (the ids that actually finished) rather than all known ids.
  3. Treat absence from TerminalSteps as 'agent still active' and read its data from DecisionSteps.

Example fix

# before
obs, reward, done, id = terminal_steps[agent_id]  # KeyError: agent still active
# after
if agent_id in terminal_steps:
    obs, reward, done, id = terminal_steps[agent_id]
else:
    active = decision_steps[agent_id]  # still running
Defensive patterns

Strategy: type-guard

Validate before calling

if agent_id not in terminal_steps.agent_id_to_index:
    # agent did not terminate this step — still active

Type guard

def is_terminal(ts, agent_id: int) -> bool:
    return agent_id in ts.agent_id_to_index

Try / catch

try:
    result = terminal_steps[agent_id]
except KeyError:
    result = None  # not done yet

Prevention

When it happens

Trigger: Indexing terminal_steps[agent_id] where that agent did not end its episode on the current step — e.g. the agent is still active and appears in DecisionSteps instead.

Common situations: Unconditionally reading terminal_steps[agent_id] for every agent every step even though only a few terminate per step; assuming an agent finished when it merely produced a decision; ids that never existed (typo, stale id).

Related errors

AI-assisted analysis of Unity-Technologies/ml-agents@3ecb446f75 (2026-09-02). Data as JSON: /api/errors/3520e85a518e7a25. Report an issue: GitHub.