huggingface/pytorch-image-models · error · ValueError

patch_coord is required for pre-patchified input.

Error message

patch_coord is required for pre-patchified input.

What it means

Gemma4ViT requires a patch_coord tensor whenever the input is pre-patchified (already tokenized patches). patch_coord supplies each patch's (x, y) position (with -1 as sentinel for padding) so positional embeddings and soft pooling can be computed; without it the model cannot place tokens and raises immediately after the ndim check.

Source

Thrown at timm/models/gemma4_vit.py:325

                patch_coord, patch_valid = self._default_patch_coord(B, H // ph, W // pw, x.device)
            x, _ = batch_patchify(x, (ph, pw), pad=False, channels_last=False)  # (B, N, C*Ph*Pw)
        elif x.ndim == 5:
            # (B, N, Ph, Pw, C) pre-patchified unflattened (NaFlex loader convention).
            # Permute channels in from last to second to produce C-Ph-Pw flat.
            x = x.permute(0, 1, 4, 2, 3).reshape(x.shape[0], x.shape[1], -1)
        elif x.ndim == 3:
            # (B, N, Ph*Pw*C) pre-patchified flat in NaFlex P-P-C layout; reinterpret as
            # (B, N, Ph, Pw, C) then permute to C-Ph-Pw flat so input_proj matches layout.
            B, N, PPC = x.shape
            C = PPC // (ph * pw)
            x = x.view(B, N, ph, pw, C).permute(0, 1, 4, 2, 3).reshape(B, N, PPC)
        else:
            raise ValueError(
                f"Expected input ndim in (3, 4, 5); got {x.ndim}."
            )

        if patch_coord is None:
            raise ValueError("patch_coord is required for pre-patchified input.")

        if patch_valid is None:
            sentinel = (patch_coord == -1).all(dim=-1)
            if sentinel.any():
                patch_valid = ~sentinel
            else:
                patch_valid = torch.ones(
                    patch_coord.shape[:2], dtype=torch.bool, device=patch_coord.device,
                )

        # Scale [0, 1] pixels to [-1, 1] (matches original Gemma4's `2 * (pixel_values - 0.5)`)
        x = 2 * (x - 0.5)
        x = self.input_proj(x.to(self.input_proj.weight.dtype))

        # Convert once to the internal (x, y) form used by rotary / pooler / table lookup.
        position_ids = patch_coord.flip(dims=(-1,))
        padding_positions = ~patch_valid
        x = x + self._position_embeddings(position_ids, padding_positions)

View on GitHub (pinned to 9a5261e31b)

Solutions

1. Compute and pass patch_coord of shape (B, N, 2) giving each patch's grid (x, y), using -1 for padding patches
2. Or feed raw images (B, C, H, W) and let the model's patch_embed generate coordinates itself
3. Persist patch_coord alongside cached patch tokens so they always travel together

Example fix

# before
out = model(patches)  # patches: (B, N, Ph*Pw*C)
# after
out = model(patches, patch_coord=coords)  # coords: (B, N, 2) int, -1 for padding

if x.ndim != 4:  # not a raw image
    assert patch_coord is not None and patch_coord.shape[:2] == x.shape[:2], \
        'pre-patchified input requires patch_coord of shape (B, N, 2)'
out = model(x, patch_coord=patch_coord)

def has_required_patch_coord(x: torch.Tensor, patch_coord) -> bool:
    return x.ndim == 4 or (patch_coord is not None and patch_coord.ndim == 3 and patch_coord.shape[-1] == 2)

try:
    out = model(patches)
except ValueError as e:
    if 'patch_coord is required' in str(e):
        out = model(patches, patch_coord=compute_coords(patches))
    else:
        raise

When it happens

Trigger: Calling model(x, patch_coord=None) (or plain model(x)) with a 3D/5D pre-patchified input; passing patch_valid but forgetting patch_coord.

Common situations: Caching patchified tokens for inference speed but dropping the coordinates array; adapting a NaFlex/Griffin-style pipeline where only patches were serialized; refactors that changed the forward signature without updating all call sites.

Related errors

• Expected input ndim in (3, 4, 5); got {x.ndim}.
• Cannot pool {N} tokens with k={k}: N must be divisible by k^
• Image size ({H}, {W}) must be divisible by (patch_size * poo
• Gemma4VitEncoder does not support classification use cases.
• output_fmt='NCHW' requires a raw image (B, C, H, W) input.

AI-assisted analysis of huggingface/pytorch-image-models@9a5261e31b (2026-08-27). Data as JSON: /api/errors/d8427e0a32d6093c. Report an issue: GitHub.