{"record":{"id":"5cc3efbf970c74f6","repo":"hankcs/HanLP","slug":"embed-dim-must-be-divisible-by-num-heads-got-emb","errorCode":null,"errorMessage":"embed_dim must be divisible by num_heads (got `embed_dim`: {self.embed_dim} and `num_heads`: {num_heads}).","messagePattern":"embed_dim must be divisible by num_heads \\(got `embed_dim`: (.+?) and `num_heads`: (.+?)\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"hanlp/components/amr/amrbart/model_interface/modeling_bart.py","lineNumber":158,"sourceCode":"class BartAttention(nn.Module):\n    \"\"\"Multi-headed attention from 'Attention Is All You Need' paper\"\"\"\n\n    def __init__(\n        self,\n        embed_dim: int,\n        num_heads: int,\n        dropout: float = 0.0,\n        is_decoder: bool = False,\n        bias: bool = True,\n    ):\n        super().__init__()\n        self.embed_dim = embed_dim\n        self.num_heads = num_heads\n        self.dropout = dropout\n        self.head_dim = embed_dim // num_heads\n\n        if (self.head_dim * num_heads) != self.embed_dim:\n            raise ValueError(\n                f\"embed_dim must be divisible by num_heads (got `embed_dim`: {self.embed_dim}\"\n                f\" and `num_heads`: {num_heads}).\"\n            )\n        self.scaling = self.head_dim**-0.5\n        self.is_decoder = is_decoder\n\n        self.k_proj = nn.Linear(embed_dim, embed_dim, bias=bias)\n        self.v_proj = nn.Linear(embed_dim, embed_dim, bias=bias)\n        self.q_proj = nn.Linear(embed_dim, embed_dim, bias=bias)\n        self.out_proj = nn.Linear(embed_dim, embed_dim, bias=bias)\n\n    def _shape(self, tensor: torch.Tensor, seq_len: int, bsz: int):\n        return tensor.view(bsz, seq_len, self.num_heads, self.head_dim).transpose(1, 2).contiguous()\n\n    def forward(\n        self,\n        hidden_states: torch.Tensor,\n        key_value_states: Optional[torch.Tensor] = None,","sourceCodeStart":140,"sourceCodeEnd":176,"githubUrl":"https://github.com/hankcs/HanLP/blob/ddb1299bddff079e447af52ec12549c50636bfa8/hanlp/components/amr/amrbart/model_interface/modeling_bart.py#L140-L176","documentation":"ScalarMix mixes exactly mixture_size tensors with learned scalar weights; if you explicitly pass initial_scalar_parameters its length must equal mixture_size. A mismatch means the init weights don't map 1:1 to the tensors being mixed and the constructor raises ValueError.","triggerScenarios":"Constructing ScalarMix(mixture_size=2, initial_scalar_parameters=[0.0, 1.0, 2.0]) or any combination where the parameter list length differs from mixture_size.","commonSituations":"Hand-writing ELMo/transformer-layer-mixing configs; increasing the number of layers mixed (e.g. adding a layer to average) without updating the initial scalar parameters; copying configs between models with different layer counts.","solutions":["Omit initial_scalar_parameters (defaults to 0.0 for each mixture element)","Set len(initial_scalar_parameters) == mixture_size","Generate the list programmatically: [0.0] * n_layers"],"exampleFix":"# before\nScalarMix(mixture_size=4, initial_scalar_parameters=[0.0, 0.0])\n# after\nScalarMix(mixture_size=4, initial_scalar_parameters=[0.0] * 4)","handlingStrategy":"validation","validationCode":"if initial_scalar_parameters is not None:\n    assert len(initial_scalar_parameters) == mixture_size, (len(initial_scalar_parameters), mixture_size)","typeGuard":null,"tryCatchPattern":"try:\n    mix = ScalarMix(mixture_size, initial_scalar_parameters=params)\nexcept ValueError:\n    mix = ScalarMix(mixture_size)  # default 0.0 init","preventionTips":["Generate init params as [0.0] * mixture_size","Omit initial_scalar_parameters unless tuning layer priors"],"tags":["hanlp","scalar-mix","config-validation","layer-mixing"],"backgroundTag":"config-validation-failed","analyzedSha":"ddb1299bddff079e447af52ec12549c50636bfa8","analyzedAt":"2026-08-27T03:36:54.287Z","schemaVersion":2},"datasetVersion":"2026-08-27T08:17:20.692Z"}