huggingface/pytorch-image-models · error · RuntimeError

features_only not implemented for Vision Transformer models.

Error message

features_only not implemented for Vision Transformer models.

What it means

CrossViT factory functions reject features_only=True. CrossViT is a dual-branch Vision Transformer (multiple input resolutions fused by cross-attention) and does not expose the feature_locations required by timm's feature extraction wrapper, so _create_crossvit raises RuntimeError up front.

Source

Thrown at timm/models/crossvit.py:494

        xs = [norm(xs[i]) for i, norm in enumerate(self.norm)]
        return xs

    def forward_head(self, xs: List[torch.Tensor], pre_logits: bool = False) -> torch.Tensor:
        xs = [x[:, 1:].mean(dim=1) for x in xs] if self.global_pool == 'avg' else [x[:, 0] for x in xs]
        xs = [self.head_drop(x) for x in xs]
        if pre_logits or isinstance(self.head[0], nn.Identity):
            return torch.cat([x for x in xs], dim=1)
        return torch.mean(torch.stack([head(xs[i]) for i, head in enumerate(self.head)], dim=0), dim=0)

    def forward(self, x):
        xs = self.forward_features(x)
        x = self.forward_head(xs)
        return x


def _create_crossvit(variant, pretrained=False, **kwargs):
    if kwargs.get('features_only', None):
        raise RuntimeError('features_only not implemented for Vision Transformer models.')

    def pretrained_filter_fn(state_dict):
        new_state_dict = {}
        for key in state_dict.keys():
            if 'pos_embed' in key or 'cls_token' in key:
                new_key = key.replace(".", "_")
            else:
                new_key = key
            new_state_dict[new_key] = state_dict[key]
        return new_state_dict

    return build_model_with_cfg(
        CrossVit,
        variant,
        pretrained,
        pretrained_filter_fn=pretrained_filter_fn,
        **kwargs,
    )

View on GitHub (pinned to 9a5261e31b)

Solutions

1. Remove features_only and use forward_intermediates for block-level outputs
2. Select a timm model with documented feature_cfgs support for feature extraction (most conv models, several ViTs)
3. Extract per-branch features manually from the model's branch modules using forward hooks

Example fix

# before
model = timm.create_model('crossvit_small_240', features_only=True)
# after
model = timm.create_model('crossvit_small_240')
feats = model.forward_intermediates(x, indices=[4, 9])

import timm
m = timm.create_model('crossvit_tiny_240')
if not hasattr(m, 'feature_info'):
    model = m  # no features_only
else:
    model = timm.create_model('crossvit_tiny_240', features_only=True)

def supports_features_only(name: str) -> bool:
    import timm
    m = timm.create_model(name)
    ok = hasattr(m, 'feature_info')
    del m
    return ok

try:
    model = timm.create_model('crossvit_15_240', features_only=True)
except RuntimeError:
    model = timm.create_model('crossvit_15_240')
    extract = lambda x: model.forward_intermediates(x, indices=[2, 5, 8])

When it happens

Trigger: Calling timm.create_model('crossvit_tiny_240', features_only=True) or any crossvit_* variant with features_only truthy.

Common situations: Feeding a uniform config (features_only=True) to many backbones in a sweep; adapting detection code written for CNN backbones to a ViT-family model without checking support.

Related errors

• features_only not implemented for Vision Transformer models.
• features_only not implemented for Vision Transformer models.
• features_only not implemented for ConvMixer models.
• Gemma4VitEncoder does not support classification use cases.
• MobileNetV5Encoder does not support classification use cases

AI-assisted analysis of huggingface/pytorch-image-models@9a5261e31b (2026-08-27). Data as JSON: /api/errors/5705885e98158d55. Report an issue: GitHub.