PaddlePaddle/PaddleOCR · error · ValueError

embed_dim must be divisible by num_heads (got `embed_dim`: {

Error message

embed_dim must be divisible by num_heads (got `embed_dim`: {self.embed_dim} and `num_heads`: {num_heads}).

What it means

The attention module in the UniMERNet head splits embed_dim into num_heads equal parts (head_dim = embed_dim // num_heads). If embed_dim is not exactly divisible by num_heads, the reshape [bsz, num_heads, tgt_len, head_dim] would lose or mix features, so the constructor fails fast with this ValueError.

Source

Thrown at ppocr/modeling/heads/rec_unimernet_head.py:554

    def __init__(
        self,
        embed_dim,
        num_heads,
        dropout: float = 0.0,
        is_decoder: bool = False,
        bias: bool = True,
        is_causal: bool = False,
        config=None,
    ):
        super().__init__()
        self.embed_dim = embed_dim
        self.num_heads = num_heads
        self.dropout = dropout
        self.head_dim = embed_dim // num_heads
        self.config = config

        if (self.head_dim * num_heads) != self.embed_dim:
            raise ValueError(
                f"embed_dim must be divisible by num_heads (got `embed_dim`: {self.embed_dim}"
                f" and `num_heads`: {num_heads})."
            )
        self.scaling = self.head_dim**-0.5
        self.is_decoder = is_decoder
        self.is_causal = is_causal

        self.k_proj = nn.Linear(embed_dim, embed_dim, bias_attr=bias)
        self.v_proj = nn.Linear(embed_dim, embed_dim, bias_attr=bias)
        self.q_proj = nn.Linear(embed_dim, embed_dim, bias_attr=bias)
        self.out_proj = nn.Linear(embed_dim, embed_dim, bias_attr=bias)

    def _shape(self, tensor, seq_len, bsz):
        return tensor.reshape([bsz, seq_len, self.num_heads, self.head_dim]).transpose(
            [0, 2, 1, 3]
        )

    def forward(

View on GitHub (pinned to 2661c7c0ef)

Solutions

1. Change num_heads in the config to a divisor of embed_dim (e.g. 384 -> 6/8/12 heads)
2. Or change embed_dim to the nearest multiple of num_heads
3. Add a config sanity check (assert d_model % num_heads == 0) in your training pipeline before model build

Example fix

// before
"d_model": 384, "num_heads": 7
// after
"d_model": 384, "num_heads": 8

assert cfg['d_model'] % cfg['num_heads'] == 0, (
    f"d_model {cfg['d_model']} not divisible by num_heads {cfg['num_heads']}")

def valid_head_dims(embed_dim: int, num_heads: int) -> bool:
    return isinstance(embed_dim, int) and isinstance(num_heads, int) and num_heads > 0 and embed_dim % num_heads == 0

When it happens

Trigger: Instantiating the head/attention with a config where d_model % num_heads != 0, e.g. embed_dim=384 with num_heads=7 (384/7 is fractional), or editing TransformerDecoder layer dims without updating head count.

Common situations: Hand-editing the UniMERNet or TBSRN-style decoder config; scaling down a model (halving dims but forgetting heads); porting configs between model variants with different head counts.

Related errors

• Head mask for a single layer should be of shape {(self.num_h
• mode can only be one of ['lite', 'large'], but received {}
• Please set inference model dir in Global.inference_model or
• "out_size" must be an integer or tuple of integers
• The hidden size ({dim}) is not a multiple of the number of a

AI-assisted analysis of PaddlePaddle/PaddleOCR@2661c7c0ef (2026-08-14). Data as JSON: /api/errors/6b21e9a5bd690f58. Report an issue: GitHub.