{"record":{"id":"279b6d6e7637099c","repo":"huggingface/pytorch-image-models","slug":"cannot-pool-n-tokens-with-k-k-n-must-be-divis","errorCode":null,"errorMessage":"Cannot pool {N} tokens with k={k}: N must be divisible by k^2={k_squared}. Both grid dimensions must be divisible by k.","messagePattern":"Cannot pool (.+?) tokens with k=(.+?): N must be divisible by k\\^2=(.+?)\\. Both grid dimensions must be divisible by k\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"timm/models/gemma4_vit.py","lineNumber":579,"sourceCode":"        self.root_hidden_size = hidden_size**0.5\n        self.pooling_kernel_size = pooling_kernel_size\n\n    def _avg_pool_by_positions(\n            self,\n            hidden_states: torch.Tensor,\n            position_ids: torch.Tensor,\n    ) -> Tuple[torch.Tensor, torch.Tensor]:\n        \"\"\"2D spatial pooling on a ``k × k`` grid (k = ``self.pooling_kernel_size``).\n\n        ``N`` patches are binned into ``k^2``-sized cells, so the pool requires\n        ``N % k^2 == 0`` (caller ensures both grid dims divide by k upstream).\n        ``position_ids`` follows the Gemma4-internal ``(x, y)`` convention.\n        \"\"\"\n        N = hidden_states.shape[1]\n        k = self.pooling_kernel_size\n        k_squared = k * k\n        if N % k_squared != 0:\n            raise ValueError(\n                f\"Cannot pool {N} tokens with k={k}: N must be divisible by k^2={k_squared}. \"\n                f\"Both grid dimensions must be divisible by k.\"\n            )\n        output_length = N // k_squared\n\n        clamped_positions = position_ids.clamp(min=0)\n        max_x = clamped_positions[..., 0].max(dim=-1, keepdim=True)[0] + 1\n        kernel_idxs = torch.div(clamped_positions, k, rounding_mode='floor')\n        kernel_idxs = kernel_idxs[..., 0] + (max_x // k) * kernel_idxs[..., 1]\n\n        weights = F.one_hot(kernel_idxs.long(), output_length).float() / k_squared\n        output = weights.transpose(1, 2) @ hidden_states.float()\n        mask = torch.logical_not((weights == 0).all(dim=1))\n        return output.to(hidden_states.dtype), mask\n\n    def forward(\n            self,\n            hidden_states: torch.Tensor,","sourceCodeStart":561,"sourceCodeEnd":597,"githubUrl":"https://github.com/huggingface/pytorch-image-models/blob/9a5261e31b3b5128526eb2658333b4c0a54464ae/timm/models/gemma4_vit.py#L561-L597","documentation":"Gemma4ViT's soft pooling (global_pool='soft') average-pools k x k blocks of patch tokens, so the token count N must be divisible by k^2 (pooling_kernel_size squared). If the sequence length does not evenly tile into k-by-k cells, pooling is geometrically undefined and ValueError is raised in _avg_pool_by_positions.","triggerScenarios":"Calling forward on a Gemma4ViT with global_pool='soft' (the default for these models) where the input image's H and W are divisible by patch_size but not by patch_size * pooling_kernel_size, e.g. 224px input with patch 16 and k=3 (needs multiples of 48).","commonSituations":"Fine-tuning at a different resolution than pretraining (e.g. 224 instead of 384) without respecting the pooling grid; NaFlex variable-resolution inputs that happen to be off-grid; test images with odd dimensions.","solutions":["Resize/crop the input so H and W are multiples of patch_size * pooling_kernel_size (e.g. multiples of 48 for patch 16, k=3)","Or switch pooling: create the model with global_pool='avg' or 'none', which does not require grid alignment","Pad the image to the next valid multiple before inference"],"exampleFix":"# before\nmodel = timm.create_model('gemma4_vit_soft', pretrained=True)\nout = model(torch.randn(1, 3, 224, 224))  # 224 % 48 != 0 -> error\n# after\nmodel = timm.create_model('gemma4_vit_soft', pretrained=True)\nout = model(torch.randn(1, 3, 240, 240))  # 240 divisible by 48","handlingStrategy":"validation","validationCode":"cell = model.patch_size[0] * model.pooling_kernel_size  # e.g. 16*3=48\nH, W = img.shape[-2:]\nif H % cell or W % cell:\n    H2, W2 = (H // cell + 1) * cell, (W // cell + 1) * cell\n    img = F.pad(img, (0, W2 - W, 0, H2 - H))\nout = model(img)","typeGuard":null,"tryCatchPattern":"try:\n    out = model(x)\nexcept ValueError as e:\n    if 'divisible by k^2' in str(e) or 'divisible' in str(e):\n        x = F.interpolate(x, size=(round_up(H, 48), round_up(W, 48)))\n        out = model(x)\n    else:\n        raise","preventionTips":["Precompute the pooling cell size (patch_size * pooling_kernel_size) in preprocessing","Snap training/inference resolutions to multiples of the cell","Offer a global_pool='avg' fallback config for arbitrary-size inputs"],"tags":["timm","gemma4-vit","pooling","image-size"],"backgroundTag":"input-size-not-divisible","analyzedSha":"9a5261e31b3b5128526eb2658333b4c0a54464ae","analyzedAt":"2026-08-27T02:34:25.417Z","schemaVersion":2},"datasetVersion":"2026-08-27T03:17:27.898Z"}