Unity-Technologies/ml-agents · error · TrainerConfigError

Unsupported reward signal configuration {d}.

Error message

Unsupported reward signal configuration {d}.

What it means

This error comes from the cattr structure hook that converts the YAML `reward_signals` mapping into RewardSignalSettings objects. It is thrown when the value being structured is not a Mapping (dict), meaning the reward_signals section was written in an unsupported shape (scalar, list, string, etc.). The hook also handles Enum-keyed selection of the correct settings class, which only works on mappings.

Source

Thrown at ml-agents/mlagents/trainers/settings.py:210

        }
        return _mapping[self]


@attr.s(auto_attribs=True)
class RewardSignalSettings:
    gamma: float = 0.99
    strength: float = 1.0
    network_settings: NetworkSettings = attr.ib(factory=NetworkSettings)

    @staticmethod
    def structure(d: Mapping, t: type) -> Any:
        """
        Helper method to structure a Dict of RewardSignalSettings class. Meant to be registered with
        cattr.register_structure_hook() and called with cattr.structure(). This is needed to handle
        the special Enum selection of RewardSignalSettings classes.
        """
        if not isinstance(d, Mapping):
            raise TrainerConfigError(f"Unsupported reward signal configuration {d}.")
        d_final: Dict[RewardSignalType, RewardSignalSettings] = {}
        for key, val in d.items():
            enum_key = RewardSignalType(key)
            t = enum_key.to_settings()
            d_final[enum_key] = strict_to_cls(val, t)
            # Checks to see if user specifying deprecated encoding_size for RewardSignals.
            # If network_settings is not specified, this updates the default hidden_units
            # to the value of encoding size. If specified, this ignores encoding size and
            # uses network_settings values.
            if "encoding_size" in val:
                logger.warning(
                    "'encoding_size' was deprecated for RewardSignals. Please use network_settings."
                )
                # If network settings was not specified, use the encoding size. Otherwise, use hidden_units
                if "network_settings" not in val:
                    d_final[enum_key].network_settings.hidden_units = val[
                        "encoding_size"
                    ]

View on GitHub (pinned to 3ecb446f75)

Solutions

1. Write reward_signals as a mapping: `reward_signals: {extrinsic: {gamma: 0.99, strength: 1.0}}`.
2. Check YAML indentation so each signal name maps to its own settings dict.
3. Validate the YAML with a parser to confirm reward_signals parses to a dict, not a scalar or list.

Example fix

# before
reward_signals: extrinsic

# after
reward_signals:
  extrinsic:
    gamma: 0.99
    strength: 1.0

rs = cfg.get('reward_signals')
if not isinstance(rs, dict):
    raise ValueError('reward_signals must be a mapping of signal name -> settings dict')

def is_reward_signal_config(v) -> bool:
    return isinstance(v, dict) and all(isinstance(k, str) for k in v)

try:
    config = TrainerSettings.structure(yaml.safe_load(f))
except TrainerConfigError as e:
    if 'reward signal' in str(e):
        fix_reward_signals_block()
    raise

When it happens

Trigger: Writing `reward_signals: extrinsic` or `reward_signals: [extrinsic]` (non-mapping) in the trainer YAML instead of a mapping of signal name -> settings.

Common situations: Hand-edited YAML where the reward_signals block was accidentally flattened, or copying an old/simplified config snippet that used shorthand syntax no longer supported.

Related errors

• Unsupported parameter randomization configuration {d}.
• There was an error decoding Config file from {config_path}.
• Error parsing yaml file. Please check for formatting errors.
• When using a recurrent network, memory size must be greater
• When using a recurrent network, memory size must be divisibl

AI-assisted analysis of Unity-Technologies/ml-agents@3ecb446f75 (2026-09-02). Data as JSON: /api/errors/ca89e413dcd2f47e. Report an issue: GitHub.