xai-org/x-algorithm · error · ValueError

key mismatch vs manifest: missing={missing} extra={extra}

Error message

key mismatch vs manifest: missing={missing} extra={extra}

What it means

load_params flattens the npz weight dict and compares its key set to manifest['params']. Missing keys exist only in the manifest; extra keys exist only in the file. Any mismatch means the artifact and its manifest describe different parameter sets, so loading aborts.

Source

Thrown at bdsm/runtime/load_backbone.py:22

import json

import numpy as np


def load_params(
    npz_path: str, manifest_path: str | None = None, verify_hashes: bool = False
) -> dict:
    z = np.load(npz_path)
    flat = {key: z[key] for key in z.files}

    if manifest_path is not None:
        with open(manifest_path) as f:
            manifest = json.load(f)
        entries = manifest["params"]
        if set(entries) != set(flat):
            missing = sorted(set(entries) - set(flat))
            extra = sorted(set(flat) - set(entries))
            raise ValueError(f"key mismatch vs manifest: missing={missing} extra={extra}")
        for key, arr in flat.items():
            meta = entries[key]
            if list(arr.shape) != meta["shape"] or str(arr.dtype) != meta["dtype"]:
                raise ValueError(
                    f"{key}: got {arr.shape}/{arr.dtype}, "
                    f"manifest says {meta['shape']}/{meta['dtype']}"
                )
            if verify_hashes:
                digest = hashlib.sha256(np.ascontiguousarray(arr).tobytes()).hexdigest()
                if digest != meta["sha256"]:
                    raise ValueError(f"{key}: sha256 mismatch")

    params: dict[str, dict[str, np.ndarray]] = {}
    for key, arr in flat.items():
        module, _, name = key.partition("/")
        params.setdefault(module, {})[name] = arr
    return params

View on GitHub (pinned to 24c60942c5)

Solutions

1. Inspect sorted(flat) vs manifest['params'] keys using the missing/extra lists in the message
2. If the npz is the source of truth, regenerate MANIFEST.json from it
3. If the manifest is authoritative, re-export the npz with the expected key layout
4. Ensure manifest and weights are exported in the same pipeline run

Example fix

# regenerate MANIFEST.json from the actual backbone.npz:
python -m bdsm.runtime.audit_manifest --backbone_dir /models/backbone --write

flat = dict(np.load(npz_path).items())
manifest_keys = set(json.load(open(manifest_path))['params'])
assert set(flat) == manifest_keys, f"drift: {set(flat) ^ manifest_keys}"

try:
    load_params(npz_path, manifest_path)
except ValueError as e:
    if 'key mismatch' in str(e):
        regenerate_manifest(npz_path)
        load_params(npz_path, manifest_path)
    else:
        raise

When it happens

Trigger: Loading a backbone.npz saved with a different module tree (renamed/added layers) than the MANIFEST.json beside it; manifest from another model version; npz containing optimizer-state keys the manifest omits.

Common situations: Model architecture change between save and manifest generation; copying a manifest from a sibling model dir; export script filtering keys differently than the manifest writer.

Related errors

• unexpected head param layout {sorted(params)} (expected {sor
• head names {unknown} are not in HEAD_ORDER — refusing to sta
• {key}: got {arr.shape}/{arr.dtype}, manifest says {meta['sha
• {key}: sha256 mismatch
• Unknown model type: {clip_model_type}. Choices: {self.MODELS

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/07c6f15860abd347. Report an issue: GitHub.