hankcs/HanLP · error · ValueError

Attention weights should be of size {(bsz * self.num_heads,

Error message

Attention weights should be of size {(bsz * self.num_heads, tgt_len, src_len)}, but is {attn_weights.size()}

What it means

At forward time ScalarMix checks (after slicing by mixture_range) that the number of tensors passed equals the mixture_size it was initialized with. If the encoder stack returns a different number of layers than the mix was built for, forward raises ValueError.

Source

Thrown at hanlp/components/amr/amrbart/model_interface/modeling_bart.py:231

            # if cross_attention save Tuple(torch.Tensor, torch.Tensor) of all cross attention key/value_states.
            # Further calls to cross_attention layer can then reuse all cross-attention
            # key/value_states (first "if" case)
            # if uni-directional self-attention (decoder) save Tuple(torch.Tensor, torch.Tensor) of
            # all previous decoder key/value_states. Further calls to uni-directional self-attention
            # can concat previous decoder key/value_states to current projected key/value_states (third "elif" case)
            # if encoder bi-directional self-attention `past_key_value` is always `None`
            past_key_value = (key_states, value_states)

        proj_shape = (bsz * self.num_heads, -1, self.head_dim)
        query_states = self._shape(query_states, tgt_len, bsz).view(*proj_shape)
        key_states = key_states.view(*proj_shape)
        value_states = value_states.view(*proj_shape)

        src_len = key_states.size(1)
        attn_weights = torch.bmm(query_states, key_states.transpose(1, 2))

        if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
            raise ValueError(
                f"Attention weights should be of size {(bsz * self.num_heads, tgt_len, src_len)}, but is"
                f" {attn_weights.size()}"
            )

        if attention_mask is not None:
            if attention_mask.size() != (bsz, 1, tgt_len, src_len):
                raise ValueError(
                    f"Attention mask should be of size {(bsz, 1, tgt_len, src_len)}, but is {attention_mask.size()}"
                )
            attn_weights = attn_weights.view(bsz, self.num_heads, tgt_len, src_len) + attention_mask
            attn_weights = attn_weights.view(bsz * self.num_heads, tgt_len, src_len)

        attn_weights = nn.functional.softmax(attn_weights, dim=-1)

        if layer_head_mask is not None:
            if layer_head_mask.size() != (self.num_heads,):
                raise ValueError(
                    f"Head mask for a single layer should be of size {(self.num_heads,)}, but is"

View on GitHub (pinned to ddb1299bdd)

Solutions

Match the tensors passed to forward with mixture_size, passing exactly the intended layer outputs
Select the specific layers before calling: ScalarMix over layers[k] list
Rebuild the ScalarMix with the correct mixture_size for your encoder

Example fix

# before
mix = ScalarMix(mixture_size=12)
out = mix(encoder_outputs.last_hidden_state)  # 1 tensor
# after
mix = ScalarMix(mixture_size=1)
out = mix([encoder_outputs.last_hidden_state])

Defensive patterns

Strategy: validation

Validate before calling

assert len(tensors) == mix.mixture_size or mix.mixture_range[1] > mix.mixture_range[0], 'tensor count mismatch'

Try / catch

try:
    out = mix(tensors)
except ValueError as e:
    raise ValueError(f'encoder returns {len(tensors)} layers but ScalarMix expects {mix.mixture_size}') from e

Prevention

Log encoder layer counts and mixture_size at setup
Select intended layers explicitly before mixing
Rebuild ScalarMix whenever the encoder config changes

When it happens

Trigger: Calling scalar_mix(tensors) with len(tensors) != mixture_size and tensors outside the mixture_range slice, e.g. feeding 12 layer outputs to a ScalarMix built for 3.

Common situations: Loading a fine-tuned model with a mismatched number of encoder layers; custom encoders returning only last_hidden_state (1 tensor) into a multi-layer mix; changing layer aggregation without rebuilding ScalarMix.

Related errors

AI-assisted analysis of hankcs/HanLP@ddb1299bdd (2026-08-27). Data as JSON: /api/errors/20ad0604d67fe2cd. Report an issue: GitHub.