{"record":{"id":"1540844d16eba79b","repo":"PaddlePaddle/PaddleOCR","slug":"the-hidden-size-dim-is-not-a-multiple-of-the-n","errorCode":null,"errorMessage":"The hidden size ({dim}) is not a multiple of the number of attention heads ({num_heads})","messagePattern":"The hidden size \\((.+?)\\) is not a multiple of the number of attention heads \\((.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"ppocr/modeling/backbones/rec_donut_swin.py","lineNumber":483,"sourceCode":"class DonutSwinDropPath(nn.Layer):\n    \"\"\"Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).\"\"\"\n\n    def __init__(self, drop_prob: Optional[float] = None) -> None:\n        super().__init__()\n        self.drop_prob = drop_prob\n\n    def forward(self, hidden_states: paddle.Tensor) -> paddle.Tensor:\n        return drop_path(hidden_states, self.drop_prob, self.training)\n\n    def extra_repr(self) -> str:\n        return \"p={}\".format(self.drop_prob)\n\n\nclass DonutSwinSelfAttention(nn.Layer):\n    def __init__(self, config, dim, num_heads, window_size):\n        super().__init__()\n        if dim % num_heads != 0:\n            raise ValueError(\n                f\"The hidden size ({dim}) is not a multiple of the number of attention heads ({num_heads})\"\n            )\n\n        self.num_attention_heads = num_heads\n        self.attention_head_size = int(dim / num_heads)\n        self.all_head_size = self.num_attention_heads * self.attention_head_size\n        self.window_size = (\n            window_size\n            if isinstance(window_size, collections.abc.Iterable)\n            else (window_size, window_size)\n        )\n        self.relative_position_bias_table = paddle.create_parameter(\n            [(2 * self.window_size[0] - 1) * (2 * self.window_size[1] - 1), num_heads],\n            dtype=\"float32\",\n        )\n        zeros_(self.relative_position_bias_table)\n\n        # get pair-wise relative position index for each token inside the window","sourceCodeStart":465,"sourceCodeEnd":501,"githubUrl":"https://github.com/PaddlePaddle/PaddleOCR/blob/2661c7c0ef5c613e8f93c6e93b2e052399f0f854/ppocr/modeling/backbones/rec_donut_swin.py#L465-L501","documentation":"DonutSwinSelfAttention splits the hidden dim evenly across attention heads (attention_head_size = dim / num_heads). If dim % num_heads != 0 the split would be fractional and the reshape impossible, so __init__ raises ValueError naming both values. This mirrors the HuggingFace transformers constraint of the same name.","triggerScenarios":"A DonutSwin config where embed_dim/hidden dim of some stage is not divisible by num_heads for that stage, e.g. depths/num_heads lists edited so a stage head count (like 4) no longer divides that stage's dim (like 30).","commonSituations":"Downsizing a pretrained config for experiments (changing embed_dim or num_heads independently), or hand-merging configs where per-stage num_heads lists get out of sync with per-stage dims.","solutions":["Make each stage's hidden size divisible by that stage's num_heads (e.g. keep dims as multiples of the head count)","Change num_heads rather than dim when shrinking: pick a divisor of the stage dim","When porting a HF transformers Donut/Swin config, copy num_heads and embed_dim together, unchanged"],"exampleFix":"# before\n# stage dim 30, heads 4 -> 30 % 4 != 0\nconfig.num_heads = [4, 4, 4, 4]\n\n# after\n# 30 divisible by 2 (or keep dim 32 with heads 4)\nconfig.num_heads = [2, 2, 2, 2]","handlingStrategy":"validation","validationCode":"for stage_dim, heads in zip(dims_per_stage, num_heads):\n    assert stage_dim % heads == 0, f'{stage_dim} not divisible by {heads} heads'","typeGuard":"def valid_head_split(dim: int, num_heads: int) -> bool:\n    return isinstance(dim, int) and isinstance(num_heads, int) and dim % num_heads == 0","tryCatchPattern":null,"preventionTips":["When downsizing configs, choose head counts as divisors of each stage's hidden dim","Port embed_dim and num_heads lists together from the source config","Add a config sanity check that walks (dims, heads) pairs before building the model"],"tags":["config","transformer","attention","donut-swin"],"backgroundTag":null,"analyzedSha":"2661c7c0ef5c613e8f93c6e93b2e052399f0f854","analyzedAt":"2026-08-14T20:17:30.180Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}