Unity-Technologies/ml-agents · error · NotImplementedError

The reward provider's update method has not been implemented

Error message

The reward provider's update method has not been implemented 

What it means

NotImplementedError raised in the abstract BaseRewardProvider.update method. It is a template-method sentinel: the base class intentionally raises so any reward provider subclass that does not override update() fails when the trainer attempts its update pass over a mini-batch. The fault is a missing override in the subclass; the message text also erroneously says 'evaluate' even though it guards the update path.

Source

Thrown at ml-agents/mlagents/trainers/torch_entities/components/reward_providers/base_reward_provider.py:71

        function drawn straight from a Buffer.
        :param mini_batch: A Dict of numpy arrays (the format used by our Buffer)
            when drawing from the update buffer.
        :return: a np.ndarray of rewards generated by the reward provider
        """
        raise NotImplementedError(
            "The reward provider's evaluate method has not been implemented "
        )

    @abstractmethod
    def update(self, mini_batch: AgentBuffer) -> Dict[str, np.ndarray]:
        """
        Update the reward for the data present in the Dict mini_batch. Use this when updating a reward
        function drawn straight from a Buffer.
        :param mini_batch: A Dict of numpy arrays (the format used by our Buffer)
            when drawing from the update buffer.
        :return: A dictionary from string to stats values
        """
        raise NotImplementedError(
            "The reward provider's update method has not been implemented "
        )

    def get_modules(self) -> Dict[str, torch.nn.Module]:
        """
        Returns a dictionary of string identifiers to the torch.nn.Modules used by
        the reward providers. This method is used for loading and saving the weights
        of the reward providers.
        """
        return {}

View on GitHub (pinned to 3ecb446f75)

Solutions

1. Implement update(self, mini_batch: AgentBuffer) -> Dict[str, np.ndarray] in the reward provider subclass, returning at least the loss statistics
2. Instantiate a concrete built-in provider (GAIL, Curiosity, RND...) instead of the base class
3. Cover the subclass with a test that performs one update step to catch missing overrides

When it happens

Trigger: Thrown at ml-agents/mlagents/trainers/torch_entities/components/reward_providers/base_reward_provider.py:71 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of Unity-Technologies/ml-agents@3ecb446f75 (2026-09-02). Data as JSON: /api/errors/aa7a18a25b4ccce9. Report an issue: GitHub.