huggingface/pytorch-image-models · error · ValueError

Patch embedding shape mismatch in {checkpoint_path}: checkpo

Error message

Patch embedding shape mismatch in {checkpoint_path}: checkpoint={tuple(embed_w.shape)}, model={tuple(embeds.proj.weight.shape)}

What it means

After conversion and optional resampling, the checkpoint patch-embedding weight must exactly match the model's embeds.proj.weight shape. A remaining mismatch (e.g. different embed dim or channel count) is fatal.

Source

Thrown at timm/models/vision_transformer.py:1530

            embed_conv_w = adapt_input_conv(embeds.in_chans, embed_conv_w)
            if embed_conv_w.shape[-2:] != embeds.patch_size:
                embed_conv_w = resample_patch_embed(
                    embed_conv_w,
                    embeds.patch_size,
                    interpolation=interpolation,
                    antialias=antialias,
                    verbose=True,
                )
            if embeds.is_linear:
                if embeds.channels_last:
                    embed_w = embed_conv_w.permute(0, 2, 3, 1).flatten(1)
                else:
                    embed_w = embed_conv_w.flatten(1)
            else:
                embed_w = embed_conv_w

        if embed_w.shape != embeds.proj.weight.shape:
            raise ValueError(
                f'Patch embedding shape mismatch in {checkpoint_path}: '
                f'checkpoint={tuple(embed_w.shape)}, model={tuple(embeds.proj.weight.shape)}')
        embeds.proj.weight.copy_(embed_w)
        if embeds.proj.bias is not None and f'{prefix}embedding/bias' in w:
            embeds.proj.bias.copy_(_n2p(w[f'{prefix}embedding/bias']))

        if embeds.cls_token is not None and f'{prefix}cls' in w:
            embeds.cls_token.copy_(_n2p(w[f'{prefix}cls'], t=False))

        pos_embed_key = (
            f'{prefix}pos_embedding' if big_vision
            else f'{prefix}Transformer/posembed_input/pos_embedding')
        if embeds.pos_embed is not None and pos_embed_key in w:
            pos_embed_w = _n2p(w[pos_embed_key], t=False)
            prefix_pos_embed = None
            if pos_embed_w.ndim == 2:
                pos_embed_w = pos_embed_w.unsqueeze(0)
            if pos_embed_w.ndim == 3:

View on GitHub (pinned to 9a5261e31b)

Solutions

1. Ensure the model variant matches the checkpoint (same embed_dim and patch_size)
2. Set in_chans=3 (or let adapt_input_conv adapt) and confirm the checkpoint stores standard channels
3. If intentional, load with strict=False outside this JAX path or pre-transform the weights to the target shape

Example fix

# before
model = vision_transformer.vit_base_patch16_224()
load_pretrained(model, 'vit_large.npz')
# after
model = vision_transformer.vit_large_patch16_224()
load_pretrained(model, 'vit_large.npz')

ew = w['embedding/kernel'].reshape(-1) if w['embedding/kernel'].ndim == 4 else w['embedding/kernel']
assert tuple(ew.shape) == tuple(model.patch_embed.proj.weight.shape), 'shape mismatch'

try:
    load_pretrained(model, path)
except ValueError as e:
    if 'Patch embedding shape mismatch' in str(e):
        log.warning('variant/checkpoint mismatch; skipping patch embed load')
    else:
        raise

When it happens

Trigger: Loading a JAX ViT checkpoint into a model with a different embed_dim, in_chans that adapt_input_conv cannot reconcile, or a patch size that resample_patch_embed did not adjust.

Common situations: Loading checkpoints into a renamed/modified architecture, or mismatched variant names (base weights into large model).

Related errors

• Unsupported patch embedding rank in {checkpoint_path}: {embe
• Cannot infer position grid from {pos_embed_w.shape[1]} token
• Unsupported position embedding shape in {checkpoint_path}: {
• The output channel {2 * self.output_size} is different from
• The output channel {self.output_size} is different from the

AI-assisted analysis of huggingface/pytorch-image-models@9a5261e31b (2026-08-27). Data as JSON: /api/errors/0861d193b877e301. Report an issue: GitHub.