{"record":{"id":"d8427e0a32d6093c","repo":"huggingface/pytorch-image-models","slug":"patch-coord-is-required-for-pre-patchified-input","errorCode":null,"errorMessage":"patch_coord is required for pre-patchified input.","messagePattern":"patch_coord is required for pre-patchified input\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"timm/models/gemma4_vit.py","lineNumber":325,"sourceCode":"                patch_coord, patch_valid = self._default_patch_coord(B, H // ph, W // pw, x.device)\n            x, _ = batch_patchify(x, (ph, pw), pad=False, channels_last=False)  # (B, N, C*Ph*Pw)\n        elif x.ndim == 5:\n            # (B, N, Ph, Pw, C) pre-patchified unflattened (NaFlex loader convention).\n            # Permute channels in from last to second to produce C-Ph-Pw flat.\n            x = x.permute(0, 1, 4, 2, 3).reshape(x.shape[0], x.shape[1], -1)\n        elif x.ndim == 3:\n            # (B, N, Ph*Pw*C) pre-patchified flat in NaFlex P-P-C layout; reinterpret as\n            # (B, N, Ph, Pw, C) then permute to C-Ph-Pw flat so input_proj matches layout.\n            B, N, PPC = x.shape\n            C = PPC // (ph * pw)\n            x = x.view(B, N, ph, pw, C).permute(0, 1, 4, 2, 3).reshape(B, N, PPC)\n        else:\n            raise ValueError(\n                f\"Expected input ndim in (3, 4, 5); got {x.ndim}.\"\n            )\n\n        if patch_coord is None:\n            raise ValueError(\"patch_coord is required for pre-patchified input.\")\n\n        if patch_valid is None:\n            sentinel = (patch_coord == -1).all(dim=-1)\n            if sentinel.any():\n                patch_valid = ~sentinel\n            else:\n                patch_valid = torch.ones(\n                    patch_coord.shape[:2], dtype=torch.bool, device=patch_coord.device,\n                )\n\n        # Scale [0, 1] pixels to [-1, 1] (matches original Gemma4's `2 * (pixel_values - 0.5)`)\n        x = 2 * (x - 0.5)\n        x = self.input_proj(x.to(self.input_proj.weight.dtype))\n\n        # Convert once to the internal (x, y) form used by rotary / pooler / table lookup.\n        position_ids = patch_coord.flip(dims=(-1,))\n        padding_positions = ~patch_valid\n        x = x + self._position_embeddings(position_ids, padding_positions)","sourceCodeStart":307,"sourceCodeEnd":343,"githubUrl":"https://github.com/huggingface/pytorch-image-models/blob/9a5261e31b3b5128526eb2658333b4c0a54464ae/timm/models/gemma4_vit.py#L307-L343","documentation":"Gemma4ViT requires a patch_coord tensor whenever the input is pre-patchified (already tokenized patches). patch_coord supplies each patch's (x, y) position (with -1 as sentinel for padding) so positional embeddings and soft pooling can be computed; without it the model cannot place tokens and raises immediately after the ndim check.","triggerScenarios":"Calling model(x, patch_coord=None) (or plain model(x)) with a 3D/5D pre-patchified input; passing patch_valid but forgetting patch_coord.","commonSituations":"Caching patchified tokens for inference speed but dropping the coordinates array; adapting a NaFlex/Griffin-style pipeline where only patches were serialized; refactors that changed the forward signature without updating all call sites.","solutions":["Compute and pass patch_coord of shape (B, N, 2) giving each patch's grid (x, y), using -1 for padding patches","Or feed raw images (B, C, H, W) and let the model's patch_embed generate coordinates itself","Persist patch_coord alongside cached patch tokens so they always travel together"],"exampleFix":"# before\nout = model(patches)  # patches: (B, N, Ph*Pw*C)\n# after\nout = model(patches, patch_coord=coords)  # coords: (B, N, 2) int, -1 for padding","handlingStrategy":"validation","validationCode":"if x.ndim != 4:  # not a raw image\n    assert patch_coord is not None and patch_coord.shape[:2] == x.shape[:2], \\\n        'pre-patchified input requires patch_coord of shape (B, N, 2)'\nout = model(x, patch_coord=patch_coord)","typeGuard":"def has_required_patch_coord(x: torch.Tensor, patch_coord) -> bool:\n    return x.ndim == 4 or (patch_coord is not None and patch_coord.ndim == 3 and patch_coord.shape[-1] == 2)","tryCatchPattern":"try:\n    out = model(patches)\nexcept ValueError as e:\n    if 'patch_coord is required' in str(e):\n        out = model(patches, patch_coord=compute_coords(patches))\n    else:\n        raise","preventionTips":["Serialize patch_coord together with cached patch tokens","Wrap pre-patchified batches in a structure that cannot be constructed without coordinates","Add a smoke test for the pre-patchified inference path"],"tags":["timm","gemma4-vit","patch-coordinates","missing-argument"],"backgroundTag":"missing-required-tensor-argument","analyzedSha":"9a5261e31b3b5128526eb2658333b4c0a54464ae","analyzedAt":"2026-08-27T02:34:25.417Z","schemaVersion":2},"datasetVersion":"2026-08-27T03:17:27.898Z"}