{"record":{"id":"6b21e9a5bd690f58","repo":"PaddlePaddle/PaddleOCR","slug":"embed-dim-must-be-divisible-by-num-heads-got-emb","errorCode":null,"errorMessage":"embed_dim must be divisible by num_heads (got `embed_dim`: {self.embed_dim} and `num_heads`: {num_heads}).","messagePattern":"embed_dim must be divisible by num_heads \\(got `embed_dim`: (.+?) and `num_heads`: (.+?)\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"ppocr/modeling/heads/rec_unimernet_head.py","lineNumber":554,"sourceCode":"    def __init__(\n        self,\n        embed_dim,\n        num_heads,\n        dropout: float = 0.0,\n        is_decoder: bool = False,\n        bias: bool = True,\n        is_causal: bool = False,\n        config=None,\n    ):\n        super().__init__()\n        self.embed_dim = embed_dim\n        self.num_heads = num_heads\n        self.dropout = dropout\n        self.head_dim = embed_dim // num_heads\n        self.config = config\n\n        if (self.head_dim * num_heads) != self.embed_dim:\n            raise ValueError(\n                f\"embed_dim must be divisible by num_heads (got `embed_dim`: {self.embed_dim}\"\n                f\" and `num_heads`: {num_heads}).\"\n            )\n        self.scaling = self.head_dim**-0.5\n        self.is_decoder = is_decoder\n        self.is_causal = is_causal\n\n        self.k_proj = nn.Linear(embed_dim, embed_dim, bias_attr=bias)\n        self.v_proj = nn.Linear(embed_dim, embed_dim, bias_attr=bias)\n        self.q_proj = nn.Linear(embed_dim, embed_dim, bias_attr=bias)\n        self.out_proj = nn.Linear(embed_dim, embed_dim, bias_attr=bias)\n\n    def _shape(self, tensor, seq_len, bsz):\n        return tensor.reshape([bsz, seq_len, self.num_heads, self.head_dim]).transpose(\n            [0, 2, 1, 3]\n        )\n\n    def forward(","sourceCodeStart":536,"sourceCodeEnd":572,"githubUrl":"https://github.com/PaddlePaddle/PaddleOCR/blob/2661c7c0ef5c613e8f93c6e93b2e052399f0f854/ppocr/modeling/heads/rec_unimernet_head.py#L536-L572","documentation":"The attention module in the UniMERNet head splits embed_dim into num_heads equal parts (head_dim = embed_dim // num_heads). If embed_dim is not exactly divisible by num_heads, the reshape [bsz, num_heads, tgt_len, head_dim] would lose or mix features, so the constructor fails fast with this ValueError.","triggerScenarios":"Instantiating the head/attention with a config where d_model % num_heads != 0, e.g. embed_dim=384 with num_heads=7 (384/7 is fractional), or editing TransformerDecoder layer dims without updating head count.","commonSituations":"Hand-editing the UniMERNet or TBSRN-style decoder config; scaling down a model (halving dims but forgetting heads); porting configs between model variants with different head counts.","solutions":["Change num_heads in the config to a divisor of embed_dim (e.g. 384 -> 6/8/12 heads)","Or change embed_dim to the nearest multiple of num_heads","Add a config sanity check (assert d_model % num_heads == 0) in your training pipeline before model build"],"exampleFix":"// before\n\"d_model\": 384, \"num_heads\": 7\n// after\n\"d_model\": 384, \"num_heads\": 8","handlingStrategy":"validation","validationCode":"assert cfg['d_model'] % cfg['num_heads'] == 0, (\n    f\"d_model {cfg['d_model']} not divisible by num_heads {cfg['num_heads']}\")","typeGuard":"def valid_head_dims(embed_dim: int, num_heads: int) -> bool:\n    return isinstance(embed_dim, int) and isinstance(num_heads, int) and num_heads > 0 and embed_dim % num_heads == 0","tryCatchPattern":null,"preventionTips":["Pick head counts from the divisor set of d_model (e.g. 384 -> {4,6,8,12,16})","Assert dims once at config load, not at model build deep in the stack"],"tags":["paddle","config","attention","shape-mismatch","constructor-validation"],"backgroundTag":null,"analyzedSha":"2661c7c0ef5c613e8f93c6e93b2e052399f0f854","analyzedAt":"2026-08-14T20:17:30.180Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}