google-research/timesfm · error · ValueError

Layer norm: {config.feedforward_norm} not supported.

Error message

Layer norm: {config.feedforward_norm} not supported.

What it means

The feedforward sublayer normalization in the Torch transformer only accepts "rms". Any other config.feedforward_norm value causes __init__ to raise this ValueError because only RMSNorm pre/post FF layers are implemented.

Source

Thrown at src/timesfm/torch/transformer.py:333

      self.pre_attn_ln = RMSNorm(num_features=config.model_dims)
      self.post_attn_ln = RMSNorm(num_features=config.model_dims)
    else:
      raise ValueError(f"Layer norm: {config.attention_norm} not supported.")

    self.attn = MultiHeadAttention(
      num_heads=config.num_heads,
      in_features=config.model_dims,
      use_per_dim_scale=True,
      use_rotary_position_embeddings=config.use_rotary_position_embeddings,
      qk_norm=config.qk_norm,
      fuse_qkv=config.fuse_qkv,
    )

    if config.feedforward_norm == "rms":
      self.pre_ff_ln = RMSNorm(num_features=config.model_dims)
      self.post_ff_ln = RMSNorm(num_features=config.model_dims)
    else:
      raise ValueError(f"Layer norm: {config.feedforward_norm} not supported.")

    self.ff0 = nn.Linear(
      in_features=config.model_dims,
      out_features=config.hidden_dims,
      bias=config.use_bias,
    )
    self.ff1 = nn.Linear(
      in_features=config.hidden_dims,
      out_features=config.model_dims,
      bias=config.use_bias,
    )
    if config.ff_activation == "relu":
      self.activation = nn.ReLU()
    elif config.ff_activation == "swish":
      self.activation = nn.SiLU()
    elif config.ff_activation == "none":
      self.activation = nn.Identity()
    else:

View on GitHub (pinned to 331c6d33cb)

Solutions

1. Set config.feedforward_norm = "rms" in the config.
2. Check spelling/casing in a hand-edited config file (comparison is case-sensitive).
3. Add an nn.LayerNorm branch in src/timesfm/torch/transformer.py __init__ if another norm is genuinely needed.

Example fix

// before
config.feedforward_norm = "layernorm"
// after
config.feedforward_norm = "rms"

if config.feedforward_norm != "rms":
    raise ValueError(f"feedforward_norm must be 'rms', got {config.feedforward_norm!r}")

def has_valid_ff_norm(config) -> bool:
    return getattr(config, "feedforward_norm", None) == "rms"

try:
    model = TimesFmTorch(config)
except ValueError as e:
    if "feedforward_norm" in str(e):
        config.feedforward_norm = "rms"
        model = TimesFmTorch(config)
    else:
        raise

When it happens

Trigger: Building the Torch transformer block with config.feedforward_norm not equal to "rms" (e.g. "layer", "layernorm", None, or a typo like "RMS").

Common situations: Same as attention_norm errors: configs copied from other architectures, manual config edits, case-sensitive typos like "RMS" instead of "rms".

Related errors

• Layer norm: {config.attention_norm} not supported.
• Activation: {config.ff_activation} not supported.
• Unsupported array shape: {x.shape}
• train_dynamic_numerical_covariates and test_dynamic_numerica
• train_dynamic_categorical_covariates and test_dynamic_catego

AI-assisted analysis of google-research/timesfm@331c6d33cb (2026-08-29). Data as JSON: /api/errors/d98f35bfcd38e668. Report an issue: GitHub.