{"record":{"id":"07c6f15860abd347","repo":"xai-org/x-algorithm","slug":"key-mismatch-vs-manifest-missing-missing-extra","errorCode":null,"errorMessage":"key mismatch vs manifest: missing={missing} extra={extra}","messagePattern":"key mismatch vs manifest: missing=(.+?) extra=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"bdsm/runtime/load_backbone.py","lineNumber":22,"sourceCode":"import json\n\nimport numpy as np\n\n\ndef load_params(\n    npz_path: str, manifest_path: str | None = None, verify_hashes: bool = False\n) -> dict:\n    z = np.load(npz_path)\n    flat = {key: z[key] for key in z.files}\n\n    if manifest_path is not None:\n        with open(manifest_path) as f:\n            manifest = json.load(f)\n        entries = manifest[\"params\"]\n        if set(entries) != set(flat):\n            missing = sorted(set(entries) - set(flat))\n            extra = sorted(set(flat) - set(entries))\n            raise ValueError(f\"key mismatch vs manifest: missing={missing} extra={extra}\")\n        for key, arr in flat.items():\n            meta = entries[key]\n            if list(arr.shape) != meta[\"shape\"] or str(arr.dtype) != meta[\"dtype\"]:\n                raise ValueError(\n                    f\"{key}: got {arr.shape}/{arr.dtype}, \"\n                    f\"manifest says {meta['shape']}/{meta['dtype']}\"\n                )\n            if verify_hashes:\n                digest = hashlib.sha256(np.ascontiguousarray(arr).tobytes()).hexdigest()\n                if digest != meta[\"sha256\"]:\n                    raise ValueError(f\"{key}: sha256 mismatch\")\n\n    params: dict[str, dict[str, np.ndarray]] = {}\n    for key, arr in flat.items():\n        module, _, name = key.partition(\"/\")\n        params.setdefault(module, {})[name] = arr\n    return params\n","sourceCodeStart":4,"sourceCodeEnd":40,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/bdsm/runtime/load_backbone.py#L4-L40","documentation":"load_params flattens the npz weight dict and compares its key set to manifest['params']. Missing keys exist only in the manifest; extra keys exist only in the file. Any mismatch means the artifact and its manifest describe different parameter sets, so loading aborts.","triggerScenarios":"Loading a backbone.npz saved with a different module tree (renamed/added layers) than the MANIFEST.json beside it; manifest from another model version; npz containing optimizer-state keys the manifest omits.","commonSituations":"Model architecture change between save and manifest generation; copying a manifest from a sibling model dir; export script filtering keys differently than the manifest writer.","solutions":["Inspect sorted(flat) vs manifest['params'] keys using the missing/extra lists in the message","If the npz is the source of truth, regenerate MANIFEST.json from it","If the manifest is authoritative, re-export the npz with the expected key layout","Ensure manifest and weights are exported in the same pipeline run"],"exampleFix":"# regenerate MANIFEST.json from the actual backbone.npz:\npython -m bdsm.runtime.audit_manifest --backbone_dir /models/backbone --write","handlingStrategy":"validation","validationCode":"flat = dict(np.load(npz_path).items())\nmanifest_keys = set(json.load(open(manifest_path))['params'])\nassert set(flat) == manifest_keys, f\"drift: {set(flat) ^ manifest_keys}\"","typeGuard":null,"tryCatchPattern":"try:\n    load_params(npz_path, manifest_path)\nexcept ValueError as e:\n    if 'key mismatch' in str(e):\n        regenerate_manifest(npz_path)\n        load_params(npz_path, manifest_path)\n    else:\n        raise","preventionTips":["Generate manifest and weights in the same export job","Never copy manifests between model directories","Add a loader smoke test to artifact publishing CI"],"tags":["python","model-loading","manifest","validation"],"backgroundTag":"manifest-schema-mismatch","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}