huggingface/pytorch-image-models · error · NotImplementedError

Gemma4VitEncoder does not support classification use cases.

Error message

Gemma4VitEncoder does not support classification use cases.

What it means

Gemma4VitEncoder is an encoder-only variant (returned with features_only wrapper usage or the Encoder class) that produces patch token sequences for downstream heads. It intentionally implements forward_head to raise NotImplementedError because there is no classifier head; calling it is a programming error.

Source

Thrown at timm/models/gemma4_vit.py:867

                x = blk(x, rope_cos, rope_sin, attn_mask=attn_mask)
            if block_callback is not None:
                block_callback(i, x)

        return x

    def forward_features(
            self,
            x: Union[torch.Tensor, Dict[str, torch.Tensor]],
            patch_coord: Optional[torch.Tensor] = None,
            patch_valid: Optional[torch.Tensor] = None,
    ) -> torch.Tensor:
        """Raw patch tokens pre-pool. Returns ``(B, N, embed_dim)``."""
        self._assert_raw_img_conformant(x if not isinstance(x, dict) else x['patches'])
        x, position_ids, padding_positions = self.patch_embed(x, patch_coord, patch_valid)
        return self._encode(x, position_ids, padding_positions)

    def forward_head(self, x: torch.Tensor, pre_logits: bool = False) -> torch.Tensor:
        raise NotImplementedError("Gemma4VitEncoder does not support classification use cases.")

    def forward(
            self,
            x: Union[torch.Tensor, Dict[str, torch.Tensor]],
            patch_coord: Optional[torch.Tensor] = None,
            patch_valid: Optional[torch.Tensor] = None,
    ) -> torch.Tensor:
        """Encode + apply the configured pool.

        Output shape depends on ``self.global_pool``:
          ``'soft'`` → ``(B, num_soft_tokens, D)``
          ``'avg'``  → ``(B, D)``
          ``'none'`` → ``(B, N, D)`` (raw patch tokens, identical to forward_features)
        """
        self._assert_raw_img_conformant(x if not isinstance(x, dict) else x['patches'])
        x, position_ids, padding_positions = self.patch_embed(x, patch_coord, patch_valid)
        x = self._encode(x, position_ids, padding_positions)

View on GitHub (pinned to 9a5261e31b)

Solutions

1. Use the full Gemma4ViT model (create_model without encoder-only flags) when a classification head is needed
2. Call encoder.forward(x) (which runs forward_features) and attach your own head on the (B, N, C) token output
3. Refactor generic pipelines to check head support before invoking forward_head

Example fix

# before
enc = timm.create_model('gemma4_vit_enc', features_only=True)
logits = enc.forward_head(enc.forward_features(x))
# after
enc = timm.create_model('gemma4_vit_enc', features_only=True)
tokens = enc(x)                 # (B, N, C)
logits = my_head(tokens.mean(1))  # custom head

if hasattr(model, 'forward_head') and type(model).forward_head is not object:
    try:
        logits = model.forward_head(feats)
    except NotImplementedError:
        logits = my_head(feats)
else:
    logits = my_head(feats)

def has_classification_head(model) -> bool:
    import timm
    return not isinstance(getattr(model, 'forward_head', None), type(None)) and 'not implemented' not in getattr(type(model).forward_head, '__doc__' or '', '')

try:
    out = model.forward_head(x)
except NotImplementedError:
    out = custom_head(x)  # tokens: (B, N, C)

When it happens

Trigger: Calling encoder.forward_head(x) directly, or passing the encoder into generic code that invokes forward_head (e.g. timm build_model_with_cfg head wiring, or custom classifiers expecting the full model API).

Common situations: Wrapping Gemma4VitEncoder in a classification pipeline that assumes the full timm Model interface; helper functions that call forward_features then forward_head generically; integration with libraries (e.g. sentence-transformers-style adapters) that probe the head API.

Related errors

• MobileNetV5Encoder does not support classification use cases
• features_only not implemented for Vision Transformer models.
• features_only not implemented for Vision Transformer models.
• features_only not implemented for ConvMixer models.
• features_only not implemented for Vision Transformer models.

AI-assisted analysis of huggingface/pytorch-image-models@9a5261e31b (2026-08-27). Data as JSON: /api/errors/d1fec3ba9d33ef66. Report an issue: GitHub.