{"record":{"id":"2745444326e20ceb","repo":"WZMIAOMIAO/deep-learning-for-image-processing","slug":"transformer-input-dimension-should-be-divisible-by","errorCode":null,"errorMessage":"Transformer input dimension should be divisible by head dimension. Got {} and {}.","messagePattern":"Transformer input dimension should be divisible by head dimension\\. Got (.+?) and (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"pytorch_classification/MobileViT/model.py","lineNumber":489,"sourceCode":"\n        if stride == 2:\n            layer = InvertedResidual(\n                in_channels=input_channel,\n                out_channels=cfg.get(\"out_channels\"),\n                stride=stride,\n                expand_ratio=cfg.get(\"mv_expand_ratio\", 4)\n            )\n\n            block.append(layer)\n            input_channel = cfg.get(\"out_channels\")\n\n        transformer_dim = cfg[\"transformer_channels\"]\n        ffn_dim = cfg.get(\"ffn_dim\")\n        num_heads = cfg.get(\"num_heads\", 4)\n        head_dim = transformer_dim // num_heads\n\n        if transformer_dim % head_dim != 0:\n            raise ValueError(\"Transformer input dimension should be divisible by head dimension. \"\n                             \"Got {} and {}.\".format(transformer_dim, head_dim))\n\n        block.append(MobileViTBlock(\n            in_channels=input_channel,\n            transformer_dim=transformer_dim,\n            ffn_dim=ffn_dim,\n            n_transformer_blocks=cfg.get(\"transformer_blocks\", 1),\n            patch_h=cfg.get(\"patch_h\", 2),\n            patch_w=cfg.get(\"patch_w\", 2),\n            dropout=cfg.get(\"dropout\", 0.1),\n            ffn_dropout=cfg.get(\"ffn_dropout\", 0.0),\n            attn_dropout=cfg.get(\"attn_dropout\", 0.1),\n            head_dim=head_dim,\n            conv_ksize=3\n        ))\n\n        return nn.Sequential(*block), input_channel\n","sourceCodeStart":471,"sourceCodeEnd":507,"githubUrl":"https://github.com/WZMIAOMIAO/deep-learning-for-image-processing/blob/1ec3fe6f374fc9969973a61f819de25658595afa/pytorch_classification/MobileViT/model.py#L471-L507","documentation":"MobileViT builds its transformer attention with head_dim = transformer_dim // num_heads, then asserts transformer_dim is divisible by head_dim. This can only fail when num_heads does not divide transformer_dim evenly (integer truncation makes head_dim wrong), so the check catches misconfigured channel/head combinations early.","triggerScenarios":"Configuring a MobileViT variant cfg where transformer_channels is not an integer multiple of num_heads, e.g. transformer_channels=200 with the default num_heads=4 leaving head_dim=50... or values like transformer_dim=100, num_heads=8 (head_dim=12, 100 % 12 != 0).","commonSituations":"Hand-editing the model config dict to shrink the model for mobile deployment while keeping default num_heads; porting configs between MobileViT variants.","solutions":["Choose transformer_channels divisible by num_heads (default 4).","Set num_heads in the cfg to a divisor of transformer_channels.","Verify head_dim = transformer_dim // num_heads leaves no remainder before building the model."],"exampleFix":"// before\ncfg = {\"transformer_channels\": 200, \"num_heads\": 8}\n// after\ncfg = {\"transformer_channels\": 200, \"num_heads\": 4}  # 200 % 50 == 0","handlingStrategy":"validation","validationCode":"def check_vit_cfg(cfg):\n    td = cfg[\"transformer_channels\"]\n    nh = cfg.get(\"num_heads\", 4)\n    assert td % nh == 0, f\"transformer_channels {td} not divisible by num_heads {nh}\"","typeGuard":"def heads_divide(transformer_dim: int, num_heads: int) -> bool:\n    return num_heads > 0 and transformer_dim % num_heads == 0","tryCatchPattern":"try:\n    model = mobile_vit_xxs(cfg)\nexcept ValueError as e:\n    print(e)\n    cfg[\"num_heads\"] = next(h for h in (4, 2, 1) if cfg[\"transformer_channels\"] % h == 0)\n    model = mobile_vit_xxs(cfg)","preventionTips":["Pick transformer_channels as a multiple of num_heads (usually 4)","Keep head_dim >= 16 for good accuracy when shrinking models","Validate cfg dicts before model construction"],"tags":["pytorch","transformer","config-validation","divisibility"],"backgroundTag":"embed-dim-head-mismatch","analyzedSha":"1ec3fe6f374fc9969973a61f819de25658595afa","analyzedAt":"2026-08-30T09:19:11.901Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}