hankcs/HanLP · error · ValueError

Head mask for a single layer should be of size {(self.num_he

Error message

Head mask for a single layer should be of size {(self.num_heads,)}, but is {layer_head_mask.size()}

What it means

TimeDistributed._reshape_tensor requires an input tensor with at least 3 dims (batch, time, features) so it can squash batch and time. A 2-D (or 1-D) tensor has no time dimension to distribute over and raises RuntimeError.

Source

Thrown at hanlp/components/amr/amrbart/model_interface/modeling_bart.py:248

        if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
            raise ValueError(
                f"Attention weights should be of size {(bsz * self.num_heads, tgt_len, src_len)}, but is"
                f" {attn_weights.size()}"
            )

        if attention_mask is not None:
            if attention_mask.size() != (bsz, 1, tgt_len, src_len):
                raise ValueError(
                    f"Attention mask should be of size {(bsz, 1, tgt_len, src_len)}, but is {attention_mask.size()}"
                )
            attn_weights = attn_weights.view(bsz, self.num_heads, tgt_len, src_len) + attention_mask
            attn_weights = attn_weights.view(bsz * self.num_heads, tgt_len, src_len)

        attn_weights = nn.functional.softmax(attn_weights, dim=-1)

        if layer_head_mask is not None:
            if layer_head_mask.size() != (self.num_heads,):
                raise ValueError(
                    f"Head mask for a single layer should be of size {(self.num_heads,)}, but is"
                    f" {layer_head_mask.size()}"
                )
            attn_weights = layer_head_mask.view(1, -1, 1, 1) * attn_weights.view(bsz, self.num_heads, tgt_len, src_len)
            attn_weights = attn_weights.view(bsz * self.num_heads, tgt_len, src_len)

        if output_attentions:
            # this operation is a bit awkward, but it's required to
            # make sure that attn_weights keeps its gradient.
            # In order to do so, attn_weights have to be reshaped
            # twice and have to be reused in the following
            attn_weights_reshaped = attn_weights.view(bsz, self.num_heads, tgt_len, src_len)
            attn_weights = attn_weights_reshaped.view(bsz * self.num_heads, tgt_len, src_len)
        else:
            attn_weights_reshaped = None

        attn_probs = nn.functional.dropout(attn_weights, p=self.dropout, training=self.training)

View on GitHub (pinned to ddb1299bdd)

Solutions

1. Keep inputs 3-D: (batch, time_steps, features) — unsqueeze(1) for single-step sequences
2. Apply the inner module directly (no TimeDistributed) for 2-D sentence-level inputs
3. Check intermediate shapes with prints/asserts when building custom architectures

Example fix

# before
out = time_dist_layer(pooled_2d)  # (B, F)
# after
out = time_dist_layer(pooled_2d.unsqueeze(1))  # (B, 1, F)

import torch
assert x.dim() >= 3, f'TimeDistributed needs (B,T,F), got {tuple(x.shape)}'

def is_sequence_tensor(x) -> bool:
    return isinstance(x, torch.Tensor) and x.dim() >= 3

try:
    out = layer(x)
except RuntimeError as e:
    if 'No dimension to distribute' in str(e):
        out = layer(x.unsqueeze(1)) if x.dim() == 2 else None
    if out is None:
        raise

When it happens

Trigger: Passing a (batch, features) tensor into a TimeDistributed-wrapped module, e.g. feeding sentence-level pooled vectors into a per-token layer.

Common situations: Mixing up sequence vs sentence-level inputs in a pipeline; a previous layer squeezed/pooled away the time dimension; feeding already-flattened (B*T, F) tensors that were meant to stay 3-D.

Related errors

• Attention mask should be of size {(bsz, 1, tgt_len, src_len)
• error
• output ({}) must be of type bool or str
• Call fit or load before evaluate.
• Unrecognized devices {devices}

AI-assisted analysis of hankcs/HanLP@ddb1299bdd (2026-08-27). Data as JSON: /api/errors/5d679959606ee460. Report an issue: GitHub.