Unity-Technologies/ml-agents · error · UnityObservationException

Observation at index={obs_index} for agent with id={agent_in

Error message

Observation at index={obs_index} for agent with id={agent_info.id} didn't match the ObservationSpec. Expected shape {expected_obs_shape} but got {agent_obs_shape}.

What it means

UnityObservationException raised by _check_observations_match_spec when an agent's observation vector's shape doesn't match the ObservationSpec declared by the Behavior for that observation index. This preempts a confusing downstream numpy error by validating shapes eagerly.

Source

Thrown at ml-agents-envs/mlagents_envs/rpc_utils.py:208

        if actual_channels > expected_channels:
            img = img[0:expected_channels, ...]
    return img


def _check_observations_match_spec(
    obs_index: int,
    observation_spec: ObservationSpec,
    agent_info_list: Collection[AgentInfoProto],
) -> None:
    """
    Check that all the observations match the expected size.
    This gives a nicer error than a cryptic numpy error later.
    """
    expected_obs_shape = tuple(observation_spec.shape)
    for agent_info in agent_info_list:
        agent_obs_shape = tuple(agent_info.observations[obs_index].shape)
        if expected_obs_shape != agent_obs_shape:
            raise UnityObservationException(
                f"Observation at index={obs_index} for agent with "
                f"id={agent_info.id} didn't match the ObservationSpec. "
                f"Expected shape {expected_obs_shape} but got {agent_obs_shape}."
            )


@timed
def _observation_to_np_array(
    obs: ObservationProto, expected_shape: Optional[Iterable[int]] = None
) -> np.ndarray:
    """
    Converts observation proto into numpy array of the appropriate size.
    :param obs: observation proto to be converted
    :param expected_shape: optional shape information, used for sanity checks.
    :return: processed numpy array of observation from environment
    """
    if expected_shape is not None:
        if list(obs.shape) != list(expected_shape):

View on GitHub (pinned to 3ecb446f75)

Solutions

Restart the UnityEnvironment so the BehaviorSpec is re-fetched from the freshly-launched environment.
Ensure agents in the scene all use the same camera resolution/stacking and sensor configuration as at initialization.
Match com.unity.ml-agents and mlagents-envs versions; rebuild the executable after scene changes.
Call env.reset() after modifying behavior parameters in the Unity editor before stepping.

Example fix

// before
env = UnityEnvironment(...)  # spec fetched
# Unity scene edited: camera height changed 84 -> 96
env.step()  # shape mismatch
// after
env.close()
env = UnityEnvironment(...)  # fresh spec matches observations

Defensive patterns

Strategy: try-catch

Validate before calling

def specs_look_fresh(env, behavior):
    spec = env.behavior_specs[behavior]
    return spec.observation_specs is not None  # spec exists; refresh env if scene changed

Try / catch

from mlagents_envs.exception import UnityObservationException

try:
    env.step()
except UnityObservationException as e:
    if "didn't match the ObservationSpec" in str(e):
        env.close()
        env = UnityEnvironment(file_name=env_path)
        env.reset()

Prevention

Never edit Unity scene/agent sensor configs while a Python session holds the env; relaunch instead.
Ensure all agents sharing a behavior name have identical observation setups.
Call env.reset() after editor-side changes.
Pin package versions across Unity and Python.

When it happens

Trigger: steps_from_proto processing agent steps where agent_info.observations[i].shape != observation_spec[i].shape for a given behavior — Unity sent data inconsistent with the spec it advertised at init.

Common situations: Changing the number of cameras/sensors or resolutions in Unity mid-run while reusing an old Python-side BehaviorSpec; stale environment instance after recompiling the Unity project; version mismatch corrupting the spec/data relationship.

Related errors

AI-assisted analysis of Unity-Technologies/ml-agents@3ecb446f75 (2026-09-02). Data as JSON: /api/errors/ce9473f507eb8660. Report an issue: GitHub.