Unity-Technologies/ml-agents · error · NotImplementedError

The reward provider's evaluate method has not been implement

Error message

The reward provider's evaluate method has not been implemented 

What it means

NotImplementedError raised in the abstract BaseRewardProvider.evaluate method. It is a template-method sentinel: the base class deliberately raises so that any reward provider subclass that does not override evaluate() fails loudly when the trainer tries to compute rewards from a buffer. The fault is a missing method implementation in the custom or mis-written subclass, not bad input data.

Source

Thrown at ml-agents/mlagents/trainers/torch_entities/components/reward_providers/base_reward_provider.py:58

    @property
    def ignore_done(self) -> bool:
        """
        If true, when the agent is done, the rewards of the next episode must be
        used to calculate the return of the current episode.
        Is used to mitigate the positive bias in rewards with no natural end.
        """
        return self._ignore_done

    @abstractmethod
    def evaluate(self, mini_batch: AgentBuffer) -> np.ndarray:
        """
        Evaluates the reward for the data present in the Dict mini_batch. Use this when evaluating a reward
        function drawn straight from a Buffer.
        :param mini_batch: A Dict of numpy arrays (the format used by our Buffer)
            when drawing from the update buffer.
        :return: a np.ndarray of rewards generated by the reward provider
        """
        raise NotImplementedError(
            "The reward provider's evaluate method has not been implemented "
        )

    @abstractmethod
    def update(self, mini_batch: AgentBuffer) -> Dict[str, np.ndarray]:
        """
        Update the reward for the data present in the Dict mini_batch. Use this when updating a reward
        function drawn straight from a Buffer.
        :param mini_batch: A Dict of numpy arrays (the format used by our Buffer)
            when drawing from the update buffer.
        :return: A dictionary from string to stats values
        """
        raise NotImplementedError(
            "The reward provider's update method has not been implemented "
        )

    def get_modules(self) -> Dict[str, torch.nn.Module]:
        """

View on GitHub (pinned to 3ecb446f75)

Solutions

1. Implement evaluate(self, mini_batch: AgentBuffer) -> np.ndarray in every custom reward provider subclass, returning one reward value per entry in the batch
2. If using built-in providers (GAIL, Curiosity, RND...), ensure you instantiate the concrete class rather than BaseRewardProvider
3. Add unit tests that call evaluate() on the subclass to catch missing overrides early

When it happens

Trigger: Thrown at ml-agents/mlagents/trainers/torch_entities/components/reward_providers/base_reward_provider.py:58 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of Unity-Technologies/ml-agents@3ecb446f75 (2026-09-02). Data as JSON: /api/errors/33ee3943c4131816. Report an issue: GitHub.