{"record":{"id":"9de807b467a70b46","repo":"google-research/timesfm","slug":"layer-norm-config-feedforward-norm-not-supporte","errorCode":null,"errorMessage":"Layer norm: {config.feedforward_norm} not supported.","messagePattern":"Layer norm: (.+?) not supported\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/timesfm/flax/transformer.py","lineNumber":316,"sourceCode":"      self.pre_attn_ln = RMSNorm(num_features=config.model_dims, rngs=rngs)\n      self.post_attn_ln = RMSNorm(num_features=config.model_dims, rngs=rngs)\n    else:\n      raise ValueError(f\"Layer norm: {config.attention_norm} not supported.\")\n\n    self.attn = MultiHeadAttention(\n      num_heads=config.num_heads,\n      in_features=config.model_dims,\n      use_per_dim_scale=True,\n      use_rotary_position_embeddings=config.use_rotary_position_embeddings,\n      qk_norm=config.qk_norm,\n      rngs=rngs,\n    )\n\n    if config.feedforward_norm == \"rms\":\n      self.pre_ff_ln = RMSNorm(num_features=config.model_dims, rngs=rngs)\n      self.post_ff_ln = RMSNorm(num_features=config.model_dims, rngs=rngs)\n    else:\n      raise ValueError(f\"Layer norm: {config.feedforward_norm} not supported.\")\n    self.ff0 = nnx.Linear(\n      in_features=config.model_dims,\n      out_features=config.hidden_dims,\n      use_bias=config.use_bias,\n      rngs=rngs,\n    )\n    self.ff1 = nnx.Linear(\n      in_features=config.hidden_dims,\n      out_features=config.model_dims,\n      use_bias=config.use_bias,\n      rngs=rngs,\n    )\n    if config.ff_activation == \"relu\":\n      self.activation = jax.nn.relu\n    elif config.ff_activation == \"swish\":\n      self.activation = jax.nn.swish\n    elif config.ff_activation == \"none\":\n      self.activation = lambda x: x","sourceCodeStart":298,"sourceCodeEnd":334,"githubUrl":"https://github.com/google-research/timesfm/blob/331c6d33cb1ac2611de3056d0ac7164aab6301eb/src/timesfm/flax/transformer.py#L298-L334","documentation":"Analogous to the attention norm check: the feed-forward sublayer only supports 'rms' (RMSNorm) for pre/post normalization, and any other config.feedforward_norm value raises ValueError in __init__.","triggerScenarios":"Constructing the transformer with config.feedforward_norm set to anything other than the exact string 'rms', e.g. 'layer_norm', 'none', 'rmsnorm', or None.","commonSituations":"Copy-pasting a config where the FF norm differs from attention_norm, misreading config docs and assuming LayerNorm is supported, or a partially populated config object where this field defaults to an unsupported value.","solutions":["Set config.feedforward_norm to 'rms'.","If a different norm is required, extend the if/elif chain in transformer.py to construct the desired norm.","Check for typos and casing ('rms' vs 'RMS'); the comparison is case-sensitive."],"exampleFix":"// before\nconfig = TransformerConfig(feedforward_norm=\"layernorm\")  # ValueError\n// after\nconfig = TransformerConfig(feedforward_norm=\"rms\")","handlingStrategy":"validation","validationCode":"if config.feedforward_norm != \"rms\":\n    raise ValueError(f\"feedforward_norm must be 'rms', got {config.feedforward_norm!r}\")","typeGuard":"def has_supported_ff_norm(cfg) -> bool:\n    return getattr(cfg, \"feedforward_norm\", None) == \"rms\"","tryCatchPattern":"try:\n    block = TransformerBlock(config)\nexcept ValueError as e:\n    if \"feedforward_norm\" in str(e) or \"Layer norm\" in str(e):\n        config.feedforward_norm = \"rms\"\n        block = TransformerBlock(config)\n    else:\n        raise","preventionTips":["Set both attention_norm and feedforward_norm to \"rms\" together.","Use Literal[\"rms\"] typing for the field.","Add a config validation pass before model construction."],"tags":["config","validation","layernorm","valueerror"],"backgroundTag":"unsupported-config-value","analyzedSha":"331c6d33cb1ac2611de3056d0ac7164aab6301eb","analyzedAt":"2026-08-29T01:04:23.138Z","schemaVersion":2},"datasetVersion":"2026-08-29T02:17:18.158Z"}