{"record":{"id":"e19fe518bde91586","repo":"xai-org/x-algorithm","slug":"tensor-name-r-checkpoint-has-shape-tuple-t-sha","errorCode":null,"errorMessage":"Tensor {name!r}: checkpoint has shape {tuple(t.shape)}, but initialized state has shape {tuple(dest.shape)}. Use 'no_loading' to skip this tensor or 'domains' to load a partial slice.","messagePattern":"Tensor (.+?): checkpoint has shape (.+?), but initialized state has shape (.+?)\\. Use 'no_loading' to skip this tensor or 'domains' to load a partial slice\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"phoenix/python/training/xai-checkpointing/xai_checkpointing/load.py","lineNumber":219,"sourceCode":"    use_zarr3: bool,\n    ts_context: ts.Context,\n    dest: jax.Array,\n    has_domain: bool,\n    tspec_transform: Callable[[dict[str, Any]], dict[str, Any]] | None,\n) -> ts.TensorStore:\n    info = ocp.type_handlers.ParamInfo(\n        name=checkpoint_name,\n        path=path / checkpoint_name,\n        parent_dir=path,\n        is_ocdbt_checkpoint=True,\n        use_zarr3=use_zarr3,\n    )\n    tspec = ocp.type_handlers.get_json_tspec_read(info, use_ocdbt=True)\n    if tspec_transform is not None:\n        tspec = tspec_transform(tspec)\n    t = ts.open(ts.Spec(tspec), open=True, context=ts_context).result()\n    if not has_domain and tuple(t.shape) != tuple(dest.shape):\n        raise ValueError(\n            f\"Tensor {name!r}: checkpoint has shape {tuple(t.shape)}, \"\n            f\"but initialized state has shape {tuple(dest.shape)}. \"\n            f\"Use 'no_loading' to skip this tensor or 'domains' to load a partial slice.\"\n        )\n    return t\n\n\ndef _read_into_shards(\n    t: ts.TensorStore,\n    array: jax.Array,\n    mask: list[bool],\n    restricted_domain: ts.IndexDomain | None = None,\n):\n    memory_kind = array.sharding.memory_kind\n    is_cpu = all(d.platform == \"cpu\" for d in array.sharding.addressable_devices)\n    assert memory_kind == \"pinned_host\" or is_cpu, (\n        f\"expected pinned_host memory, got {memory_kind!r}\"\n    )","sourceCodeStart":201,"sourceCodeEnd":237,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/python/training/xai-checkpointing/xai_checkpointing/load.py#L201-L237","documentation":"_open_tensor opens a tensor from the checkpoint and, when no domain restriction is given, requires its shape to equal the destination (initialized state) tensor's shape. A mismatch raises this ValueError suggesting 'no_loading' or 'domains'.","triggerScenarios":"load_checkpoint where the model was initialized with different shapes than saved — e.g. vocab_size, mesh/sharding changes, or num_layers changed in the config while reusing an old checkpoint.","commonSituations":"Resuming after a config change that alters a tensor's axis size, loading a sharded checkpoint into a differently-shaped mesh, or model architecture version changes.","solutions":["Revert the config change so shapes match the checkpoint","Add the tensor to no_loading to skip restoring it","Use domains to load the overlapping slice if sizes changed intentionally (e.g. extended vocab)","Re-save the checkpoint after the architecture change so shapes align"],"exampleFix":"# before\nload_checkpoint(path, host_state, no_loading=set())\n# after\nload_checkpoint(path, host_state, no_loading={\"embedding\"})  # vocab resized","handlingStrategy":"validation","validationCode":"# compare shapes before loading\nfor name, dest in state.items():\n    ckpt_shape = checkpoint_index[name].shape  # from metadata\n    if tuple(ckpt_shape) != tuple(dest.shape):\n        print(f\"shape mismatch: {name} {ckpt_shape} vs {dest.shape}\")","typeGuard":null,"tryCatchPattern":"try:\n    load_checkpoint(...)\nexcept ValueError as e:\n    if \"checkpoint has shape\" in str(e):\n        load_checkpoint(..., no_loading=offending_tensors)","preventionTips":["Record the config/schema hash with each checkpoint","Add a pre-load shape check from checkpoint metadata"],"tags":["checkpoint","shape-mismatch","model-loading"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}