{"record":{"id":"c0cf6f2ea70912e6","repo":"google-research/timesfm","slug":"memory-dimension-self-in-features-must-be-divi","errorCode":null,"errorMessage":"Memory dimension ({self.in_features}) must be divisible by 'num_heads' heads ({self.num_heads}).","messagePattern":"Memory dimension \\((.+?)\\) must be divisible by 'num_heads' heads \\((.+?)\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/timesfm/torch/transformer.py","lineNumber":194,"sourceCode":"    *,\n    use_per_dim_scale: bool = True,\n    use_rotary_position_embeddings: bool = True,\n    use_bias: bool = False,\n    attention_fn: Callable[..., torch.Tensor] = _torch_dot_product_attention,\n    qk_norm: str = \"rms\",\n    fuse_qkv: bool = False,\n  ):\n    super().__init__()\n    self.num_heads = num_heads\n    self.in_features = in_features\n    self.head_dim = in_features // num_heads\n    self.use_bias = use_bias\n    self.attention_fn = attention_fn\n    self.qk_norm = qk_norm\n    self.fuse_qkv = fuse_qkv\n\n    if self.in_features % self.num_heads != 0:\n      raise ValueError(\n        f\"Memory dimension ({self.in_features}) must be divisible by \"\n        f\"'num_heads' heads ({self.num_heads}).\"\n      )\n\n    if self.fuse_qkv:\n      self.qkv_proj = nn.Linear(self.in_features, 3 * self.in_features, bias=use_bias)\n    else:\n      self.query = nn.Linear(self.in_features, self.in_features, bias=use_bias)\n      self.key = nn.Linear(self.in_features, self.in_features, bias=use_bias)\n      self.value = nn.Linear(self.in_features, self.in_features, bias=use_bias)\n    self.out = nn.Linear(self.in_features, self.in_features, bias=use_bias)\n\n    if self.qk_norm == \"rms\":\n      self.query_ln = RMSNorm(self.head_dim)\n      self.key_ln = RMSNorm(self.head_dim)\n    else:\n      self.query_ln = nn.Identity()\n      self.key_ln = nn.Identity()","sourceCodeStart":176,"sourceCodeEnd":212,"githubUrl":"https://github.com/google-research/timesfm/blob/331c6d33cb1ac2611de3056d0ac7164aab6301eb/src/timesfm/torch/transformer.py#L176-L212","documentation":"Multi-head attention splits the memory/key-value dimension evenly across heads: each head gets in_features // num_heads features. If in_features % num_heads != 0 the split is impossible, so __init__ raises ValueError. In the reference config, model_dims=1280 with num_heads=16.","triggerScenarios":"Constructing the attention layer (via TransformerConfig) where in_features is not divisible by num_heads — e.g. model_dims=1000 with num_heads=16, or an arbitrary experimental num_heads like 100.","commonSituations":"Customizing model_dims or num_heads for smaller/faster models with incompatible values; reusing a num_heads from another config; typos in num_heads.","solutions":["Choose num_heads that divides in_features exactly (model_dims=1280 works with 8, 16, 20, 32 heads).","When changing model_dims, pick a highly composite number for head-count flexibility.","Validate at config time: assert model_dims % num_heads == 0 before building.","Match the reference config (model_dims=1280, num_heads=16) unless there is a specific need."],"exampleFix":"// before\nTransformerConfig(model_dims=1000, num_heads=16)  # ValueError: 1000 % 16 != 0\n// after\nTransformerConfig(model_dims=1024, num_heads=16)  # 1024 / 16 = 64 per head","handlingStrategy":"validation","validationCode":"if model_dims % num_heads != 0:\n    raise ValueError(f\"model_dims={model_dims} not divisible by num_heads={num_heads}\")","typeGuard":null,"tryCatchPattern":"try:\n    attn = AttentionLayer(in_features=model_dims, num_heads=num_heads, ...)\nexcept ValueError as e:\n    if \"divisible\" in str(e):\n        model_dims = (model_dims // num_heads + 1) * num_heads\n        attn = AttentionLayer(in_features=model_dims, num_heads=num_heads, ...)\n    else:\n        raise","preventionTips":["Pick num_heads as a divisor of model_dims; prefer highly composite dims (1280, 1024).","Assert model_dims % num_heads == 0 wherever TransformerConfig is built.","Start from the reference config (1280 dims, 16 heads) and change one knob at a time.","Document the divisibility constraint next to custom config factories."],"tags":["configuration","value-error","attention","dimension-mismatch","torch"],"backgroundTag":"dimension-not-divisible","analyzedSha":"331c6d33cb1ac2611de3056d0ac7164aab6301eb","analyzedAt":"2026-08-29T01:04:23.138Z","schemaVersion":2},"datasetVersion":"2026-08-29T02:17:18.158Z"}