{"record":{"id":"4fa3dcbf53508828","repo":"Unity-Technologies/ml-agents","slug":"metadata-not-found-resuming-from-an-incompatible","errorCode":null,"errorMessage":"Metadata not found, resuming from an incompatible version of ML-Agents.","messagePattern":"Metadata not found, resuming from an incompatible version of ML-Agents\\.","errorType":"exception","errorClass":"TrainerError","httpStatus":null,"severity":"error","filePath":"ml-agents/mlagents/trainers/training_status.py","lineNumber":83,"sourceCode":"    def load_state(path: str) -> None:\n        \"\"\"\n        Load a JSON file that contains saved state.\n        :param path: Path to the JSON file containing the state.\n        \"\"\"\n        try:\n            with open(path) as f:\n                loaded_dict = json.load(f)\n            # Compare the metadata\n            _metadata = loaded_dict[StatusType.STATS_METADATA.value]\n            StatusMetaData.from_dict(_metadata).check_compatibility(StatusMetaData())\n            # Update saved state.\n            GlobalTrainingStatus.saved_state.update(loaded_dict)\n        except FileNotFoundError:\n            logger.warning(\n                \"Training status file not found. Not all functions will resume properly.\"\n            )\n        except KeyError:\n            raise TrainerError(\n                \"Metadata not found, resuming from an incompatible version of ML-Agents.\"\n            )\n\n    @staticmethod\n    def save_state(path: str) -> None:\n        \"\"\"\n        Save a JSON file that contains saved state.\n        :param path: Path to the JSON file containing the state.\n        \"\"\"\n        GlobalTrainingStatus.saved_state[\n            StatusType.STATS_METADATA.value\n        ] = StatusMetaData().to_dict()\n        with open(path, \"w\") as f:\n            json.dump(GlobalTrainingStatus.saved_state, f, indent=4)\n\n    @staticmethod\n    def set_parameter_state(category: str, key: StatusType, value: Any) -> None:\n        \"\"\"","sourceCodeStart":65,"sourceCodeEnd":101,"githubUrl":"https://github.com/Unity-Technologies/ml-agents/blob/3ecb446f75d1e7400eb404c562dc005d3164cffc/ml-agents/mlagents/trainers/training_status.py#L65-L101","documentation":"GlobalTrainingStatus.load_state() restores trainer metadata from a saved 'training_status.json' produced by a previous run. The loaded dict must contain a 'metadata' key; if it doesn't, ML-Agents concludes the checkpoint was written by an incompatible version and raises TrainerError instead of silently resuming with partial state.","triggerScenarios":"Calling GlobalTrainingStatus.load_state(path) (via mlagents-learn --resume) on a training_status.json whose top-level dict lacks the 'metadata' key — i.e. the file was saved by an older ML-Agents release with a different state schema, or the file was hand-edited/corrupted and 'metadata' was removed.","commonSituations":"Resuming a run that was checkpointed with a much older ML-Agents version after upgrading the package; copying a training_status.json from a different project/branch; manual edits to the status file.","solutions":["Regenerate the run with the ML-Agents version you are using now — start training fresh rather than resuming the old checkpoint.","Check out/pip-install the ML-Agents version that originally created the checkpoint, resume it there, then upgrade.","Inspect training_status.json and confirm/add a top-level \"metadata\" key if you know the file is valid and merely edited.","Delete or archive the stale training_status.json so load_state fails cleanly at start rather than mid-run."],"exampleFix":"// before (resume old checkpoint)\nmlagents-learn config.yaml --run-id=old-run --resume\n// raises: Metadata not found...\n// after (fresh run with current version)\nmlagents-learn config.yaml --run-id=new-run","handlingStrategy":"validation","validationCode":"import json\nfrom mlagents.trainers.exception import TrainerError\n\ndef validate_status_file(path):\n    with open(path) as f:\n        state = json.load(f)\n    if \"metadata\" not in state:\n        raise ValueError(f\"{path} lacks 'metadata'; incompatible with this ML-Agents version\")\n    return state","typeGuard":"def is_compatible_status(state: dict) -> bool:\n    return isinstance(state, dict) and \"metadata\" in state","tryCatchPattern":"from mlagents.trainers.exception import TrainerError\ntry:\n    GlobalTrainingStatus.load_state(path)\nexcept TrainerError as e:\n    if \"Metadata not found\" in str(e):\n        logger.warning(\"Checkpoint from incompatible ML-Agents version; starting fresh run\")\n    else:\n        raise","preventionTips":["Pin the same mlagents version for checkpointing and resuming.","Keep training_status.json out of manual edit workflows.","Store the mlagents version alongside run artifacts and check before resuming.","Archive old checkpoints instead of resuming across major version jumps."],"tags":["python","ml-agents","resume","checkpoint-incompatible"],"backgroundTag":"incompatible-checkpoint-version","analyzedSha":"3ecb446f75d1e7400eb404c562dc005d3164cffc","analyzedAt":"2026-09-02T16:33:12.832Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T21:17:11.164Z"}