google-research/timesfm · error · ValueError

Layer norm: {config.feedforward_norm} not supported.

Error message

Layer norm: {config.feedforward_norm} not supported.

What it means

Analogous to the attention norm check: the feed-forward sublayer only supports 'rms' (RMSNorm) for pre/post normalization, and any other config.feedforward_norm value raises ValueError in __init__.

Source

Thrown at src/timesfm/flax/transformer.py:316

      self.pre_attn_ln = RMSNorm(num_features=config.model_dims, rngs=rngs)
      self.post_attn_ln = RMSNorm(num_features=config.model_dims, rngs=rngs)
    else:
      raise ValueError(f"Layer norm: {config.attention_norm} not supported.")

    self.attn = MultiHeadAttention(
      num_heads=config.num_heads,
      in_features=config.model_dims,
      use_per_dim_scale=True,
      use_rotary_position_embeddings=config.use_rotary_position_embeddings,
      qk_norm=config.qk_norm,
      rngs=rngs,
    )

    if config.feedforward_norm == "rms":
      self.pre_ff_ln = RMSNorm(num_features=config.model_dims, rngs=rngs)
      self.post_ff_ln = RMSNorm(num_features=config.model_dims, rngs=rngs)
    else:
      raise ValueError(f"Layer norm: {config.feedforward_norm} not supported.")
    self.ff0 = nnx.Linear(
      in_features=config.model_dims,
      out_features=config.hidden_dims,
      use_bias=config.use_bias,
      rngs=rngs,
    )
    self.ff1 = nnx.Linear(
      in_features=config.hidden_dims,
      out_features=config.model_dims,
      use_bias=config.use_bias,
      rngs=rngs,
    )
    if config.ff_activation == "relu":
      self.activation = jax.nn.relu
    elif config.ff_activation == "swish":
      self.activation = jax.nn.swish
    elif config.ff_activation == "none":
      self.activation = lambda x: x

View on GitHub (pinned to 331c6d33cb)

Solutions

1. Set config.feedforward_norm to 'rms'.
2. If a different norm is required, extend the if/elif chain in transformer.py to construct the desired norm.
3. Check for typos and casing ('rms' vs 'RMS'); the comparison is case-sensitive.

Example fix

// before
config = TransformerConfig(feedforward_norm="layernorm")  # ValueError
// after
config = TransformerConfig(feedforward_norm="rms")

if config.feedforward_norm != "rms":
    raise ValueError(f"feedforward_norm must be 'rms', got {config.feedforward_norm!r}")

def has_supported_ff_norm(cfg) -> bool:
    return getattr(cfg, "feedforward_norm", None) == "rms"

try:
    block = TransformerBlock(config)
except ValueError as e:
    if "feedforward_norm" in str(e) or "Layer norm" in str(e):
        config.feedforward_norm = "rms"
        block = TransformerBlock(config)
    else:
        raise

When it happens

Trigger: Constructing the transformer with config.feedforward_norm set to anything other than the exact string 'rms', e.g. 'layer_norm', 'none', 'rmsnorm', or None.

Common situations: Copy-pasting a config where the FF norm differs from attention_norm, misreading config docs and assuming LayerNorm is supported, or a partially populated config object where this field defaults to an unsupported value.

Related errors

• Layer norm: {config.attention_norm} not supported.
• Activation: {config.activation} not supported.
• Output dims must be a multiple of 4: {config.output_dims} %
• Activation: {config.ff_activation} not supported.
• Memory dimension ({self.qkv_features}) must be divisible by

AI-assisted analysis of google-research/timesfm@331c6d33cb (2026-08-29). Data as JSON: /api/errors/9de807b467a70b46. Report an issue: GitHub.