{"record":{"id":"d1fec3ba9d33ef66","repo":"huggingface/pytorch-image-models","slug":"gemma4vitencoder-does-not-support-classification-u","errorCode":null,"errorMessage":"Gemma4VitEncoder does not support classification use cases.","messagePattern":"Gemma4VitEncoder does not support classification use cases\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"timm/models/gemma4_vit.py","lineNumber":867,"sourceCode":"                x = blk(x, rope_cos, rope_sin, attn_mask=attn_mask)\n            if block_callback is not None:\n                block_callback(i, x)\n\n        return x\n\n    def forward_features(\n            self,\n            x: Union[torch.Tensor, Dict[str, torch.Tensor]],\n            patch_coord: Optional[torch.Tensor] = None,\n            patch_valid: Optional[torch.Tensor] = None,\n    ) -> torch.Tensor:\n        \"\"\"Raw patch tokens pre-pool. Returns ``(B, N, embed_dim)``.\"\"\"\n        self._assert_raw_img_conformant(x if not isinstance(x, dict) else x['patches'])\n        x, position_ids, padding_positions = self.patch_embed(x, patch_coord, patch_valid)\n        return self._encode(x, position_ids, padding_positions)\n\n    def forward_head(self, x: torch.Tensor, pre_logits: bool = False) -> torch.Tensor:\n        raise NotImplementedError(\"Gemma4VitEncoder does not support classification use cases.\")\n\n    def forward(\n            self,\n            x: Union[torch.Tensor, Dict[str, torch.Tensor]],\n            patch_coord: Optional[torch.Tensor] = None,\n            patch_valid: Optional[torch.Tensor] = None,\n    ) -> torch.Tensor:\n        \"\"\"Encode + apply the configured pool.\n\n        Output shape depends on ``self.global_pool``:\n          ``'soft'`` → ``(B, num_soft_tokens, D)``\n          ``'avg'``  → ``(B, D)``\n          ``'none'`` → ``(B, N, D)`` (raw patch tokens, identical to forward_features)\n        \"\"\"\n        self._assert_raw_img_conformant(x if not isinstance(x, dict) else x['patches'])\n        x, position_ids, padding_positions = self.patch_embed(x, patch_coord, patch_valid)\n        x = self._encode(x, position_ids, padding_positions)\n","sourceCodeStart":849,"sourceCodeEnd":885,"githubUrl":"https://github.com/huggingface/pytorch-image-models/blob/9a5261e31b3b5128526eb2658333b4c0a54464ae/timm/models/gemma4_vit.py#L849-L885","documentation":"Gemma4VitEncoder is an encoder-only variant (returned with features_only wrapper usage or the Encoder class) that produces patch token sequences for downstream heads. It intentionally implements forward_head to raise NotImplementedError because there is no classifier head; calling it is a programming error.","triggerScenarios":"Calling encoder.forward_head(x) directly, or passing the encoder into generic code that invokes forward_head (e.g. timm build_model_with_cfg head wiring, or custom classifiers expecting the full model API).","commonSituations":"Wrapping Gemma4VitEncoder in a classification pipeline that assumes the full timm Model interface; helper functions that call forward_features then forward_head generically; integration with libraries (e.g. sentence-transformers-style adapters) that probe the head API.","solutions":["Use the full Gemma4ViT model (create_model without encoder-only flags) when a classification head is needed","Call encoder.forward(x) (which runs forward_features) and attach your own head on the (B, N, C) token output","Refactor generic pipelines to check head support before invoking forward_head"],"exampleFix":"# before\nenc = timm.create_model('gemma4_vit_enc', features_only=True)\nlogits = enc.forward_head(enc.forward_features(x))\n# after\nenc = timm.create_model('gemma4_vit_enc', features_only=True)\ntokens = enc(x)                 # (B, N, C)\nlogits = my_head(tokens.mean(1))  # custom head","handlingStrategy":"type-guard","validationCode":"if hasattr(model, 'forward_head') and type(model).forward_head is not object:\n    try:\n        logits = model.forward_head(feats)\n    except NotImplementedError:\n        logits = my_head(feats)\nelse:\n    logits = my_head(feats)","typeGuard":"def has_classification_head(model) -> bool:\n    import timm\n    return not isinstance(getattr(model, 'forward_head', None), type(None)) and 'not implemented' not in getattr(type(model).forward_head, '__doc__' or '', '')","tryCatchPattern":"try:\n    out = model.forward_head(x)\nexcept NotImplementedError:\n    out = custom_head(x)  # tokens: (B, N, C)","preventionTips":["Check model type (encoder vs full) before calling head methods","Route encoder outputs through your own task head by design","Use create_model without features_only when built-in classification is required"],"tags":["timm","gemma4-vit","encoder-only","unsupported-operation"],"backgroundTag":"encoder-has-no-classification-head","analyzedSha":"9a5261e31b3b5128526eb2658333b4c0a54464ae","analyzedAt":"2026-08-27T02:34:25.417Z","schemaVersion":2},"datasetVersion":"2026-08-27T03:17:27.898Z"}