{"record":{"id":"35d11ecc0b14519a","repo":"google-research/timesfm","slug":"layer-norm-config-attention-norm-not-supported-35d11e","errorCode":null,"errorMessage":"Layer norm: {config.attention_norm} not supported.","messagePattern":"Layer norm: (.+?) not supported\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/timesfm/torch/transformer.py","lineNumber":318,"sourceCode":"    )\n\n    x = x.reshape(b, n_patches, self.in_features)\n    out = self.out(x)\n    return out, decode_cache\n\n\nclass Transformer(nn.Module):\n  \"\"\"Classic Transformer used in TimesFM.\"\"\"\n\n  def __init__(self, config: configs.TransformerConfig):\n    super().__init__()\n    self.config = config\n\n    if config.attention_norm == \"rms\":\n      self.pre_attn_ln = RMSNorm(num_features=config.model_dims)\n      self.post_attn_ln = RMSNorm(num_features=config.model_dims)\n    else:\n      raise ValueError(f\"Layer norm: {config.attention_norm} not supported.\")\n\n    self.attn = MultiHeadAttention(\n      num_heads=config.num_heads,\n      in_features=config.model_dims,\n      use_per_dim_scale=True,\n      use_rotary_position_embeddings=config.use_rotary_position_embeddings,\n      qk_norm=config.qk_norm,\n      fuse_qkv=config.fuse_qkv,\n    )\n\n    if config.feedforward_norm == \"rms\":\n      self.pre_ff_ln = RMSNorm(num_features=config.model_dims)\n      self.post_ff_ln = RMSNorm(num_features=config.model_dims)\n    else:\n      raise ValueError(f\"Layer norm: {config.feedforward_norm} not supported.\")\n\n    self.ff0 = nn.Linear(\n      in_features=config.model_dims,","sourceCodeStart":300,"sourceCodeEnd":336,"githubUrl":"https://github.com/google-research/timesfm/blob/331c6d33cb1ac2611de3056d0ac7164aab6301eb/src/timesfm/torch/transformer.py#L300-L336","documentation":"The Torch transformer block only supports RMS normalization for attention layers. At construction time, if config.attention_norm is anything other than \"rms\", the block raises this ValueError because no pre/post-attention LayerNorm module can be created. The f-string interpolation in the source actually shows the raw {config.attention_norm} placeholder text when raised.","triggerScenarios":"Constructing a Torch transformer block (e.g. TimesFmTorch or its stack of blocks) with config.attention_norm set to anything other than \"rms\" (e.g. \"layer\", \"layernorm\", or a None value).","commonSituations":"Porting configs from other models (which use standard LayerNorm) into TimesFM, hand-editing config dicts/JSON, or copying a config from the JAX/Flax implementation that allows other norm types.","solutions":["Set config.attention_norm = \"rms\" in the TimesFM config before constructing the model.","If the config was loaded from a checkpoint/JSON, inspect and fix the attention_norm field to \"rms\".","If you need LayerNorm, extend the __init__ in src/timesfm/torch/transformer.py to add an nn.LayerNorm branch."],"exampleFix":"// before\nconfig = TimesFmConfig(model_dims=1280, attention_norm=\"layer\")\nmodel = TimesFmTorch(config)  # raises ValueError\n// after\nconfig = TimesFmConfig(model_dims=1280, attention_norm=\"rms\")\nmodel = TimesFmTorch(config)  # ok","handlingStrategy":"validation","validationCode":"if config.attention_norm != \"rms\":\n    raise ValueError(f\"attention_norm must be 'rms', got {config.attention_norm!r}\")","typeGuard":"def has_valid_attention_norm(config) -> bool:\n    return getattr(config, \"attention_norm\", None) == \"rms\"","tryCatchPattern":"try:\n    model = TimesFmTorch(config)\nexcept ValueError as e:\n    if \"attention_norm\" in str(e):\n        config.attention_norm = \"rms\"\n        model = TimesFmTorch(config)\n    else:\n        raise","preventionTips":["Always build configs via TimesFmConfig defaults rather than raw dicts","Validate config fields against allowed values before instantiating the model","Watch for case-sensitivity: use lowercase \"rms\""],"tags":["python","configuration","valueerror"],"backgroundTag":"unsupported-config-value","analyzedSha":"331c6d33cb1ac2611de3056d0ac7164aab6301eb","analyzedAt":"2026-08-29T01:04:23.138Z","schemaVersion":2},"datasetVersion":"2026-08-29T02:17:18.158Z"}