PaddlePaddle/PaddleOCR · error · ValueError

The hidden size ({dim}) is not a multiple of the number of a

Error message

The hidden size ({dim}) is not a multiple of the number of attention heads ({num_heads})

What it means

DonutSwinSelfAttention splits the hidden dim evenly across attention heads (attention_head_size = dim / num_heads). If dim % num_heads != 0 the split would be fractional and the reshape impossible, so __init__ raises ValueError naming both values. This mirrors the HuggingFace transformers constraint of the same name.

Source

Thrown at ppocr/modeling/backbones/rec_donut_swin.py:483

class DonutSwinDropPath(nn.Layer):
    """Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks)."""

    def __init__(self, drop_prob: Optional[float] = None) -> None:
        super().__init__()
        self.drop_prob = drop_prob

    def forward(self, hidden_states: paddle.Tensor) -> paddle.Tensor:
        return drop_path(hidden_states, self.drop_prob, self.training)

    def extra_repr(self) -> str:
        return "p={}".format(self.drop_prob)


class DonutSwinSelfAttention(nn.Layer):
    def __init__(self, config, dim, num_heads, window_size):
        super().__init__()
        if dim % num_heads != 0:
            raise ValueError(
                f"The hidden size ({dim}) is not a multiple of the number of attention heads ({num_heads})"
            )

        self.num_attention_heads = num_heads
        self.attention_head_size = int(dim / num_heads)
        self.all_head_size = self.num_attention_heads * self.attention_head_size
        self.window_size = (
            window_size
            if isinstance(window_size, collections.abc.Iterable)
            else (window_size, window_size)
        )
        self.relative_position_bias_table = paddle.create_parameter(
            [(2 * self.window_size[0] - 1) * (2 * self.window_size[1] - 1), num_heads],
            dtype="float32",
        )
        zeros_(self.relative_position_bias_table)

        # get pair-wise relative position index for each token inside the window

View on GitHub (pinned to 2661c7c0ef)

Solutions

1. Make each stage's hidden size divisible by that stage's num_heads (e.g. keep dims as multiples of the head count)
2. Change num_heads rather than dim when shrinking: pick a divisor of the stage dim
3. When porting a HF transformers Donut/Swin config, copy num_heads and embed_dim together, unchanged

Example fix

# before
# stage dim 30, heads 4 -> 30 % 4 != 0
config.num_heads = [4, 4, 4, 4]

# after
# 30 divisible by 2 (or keep dim 32 with heads 4)
config.num_heads = [2, 2, 2, 2]

for stage_dim, heads in zip(dims_per_stage, num_heads):
    assert stage_dim % heads == 0, f'{stage_dim} not divisible by {heads} heads'

def valid_head_split(dim: int, num_heads: int) -> bool:
    return isinstance(dim, int) and isinstance(num_heads, int) and dim % num_heads == 0

When it happens

Trigger: A DonutSwin config where embed_dim/hidden dim of some stage is not divisible by num_heads for that stage, e.g. depths/num_heads lists edited so a stage head count (like 4) no longer divides that stage's dim (like 30).

Common situations: Downsizing a pretrained config for experiments (changing embed_dim or num_heads independently), or hand-merging configs where per-stage num_heads lists get out of sync with per-stage dims.

Related errors

• invalid layer type {layer_type}
• embed_dim must be divisible by num_heads (got `embed_dim`: {
• RecResizeImg.image_shape is required in rec inference.yml
• Unexpected recognition channels: ${String(channels)}
• OCR pipeline config text must decode to an object.

AI-assisted analysis of PaddlePaddle/PaddleOCR@2661c7c0ef (2026-08-14). Data as JSON: /api/errors/1540844d16eba79b. Report an issue: GitHub.