What causes "Layer norm: {config.attention_norm} not supported." in google-research/timesfm?

Set config.attention_norm to 'rms' (exact string, lowercase).

google-research/timesfm · error · ValueError

Layer norm: {config.attention_norm} not supported.

Error message

Layer norm: {config.attention_norm} not supported.

What it means

The transformer block only implements 'rms' (RMSNorm) for attention pre/post layer normalization; any other config.attention_norm value raises this ValueError in __init__. Unlike a flexible enum, there is exactly one accepted value in this code path.

Source

Thrown at src/timesfm/flax/transformer.py:301

      deterministic=deterministic,
      module=self if sow_weights else None,
    )
    # back to the original inputs dimensions
    out = self.out(x)
    return out, decode_cache


class Transformer(nnx.Module):
  """Classic Transformer used in TimesFM."""

  def __init__(self, config: TransformerConfig, *, rngs=nnx.Rngs(42)):
    self.config = config

    if config.attention_norm == "rms":
      self.pre_attn_ln = RMSNorm(num_features=config.model_dims, rngs=rngs)
      self.post_attn_ln = RMSNorm(num_features=config.model_dims, rngs=rngs)
    else:
      raise ValueError(f"Layer norm: {config.attention_norm} not supported.")

    self.attn = MultiHeadAttention(
      num_heads=config.num_heads,
      in_features=config.model_dims,
      use_per_dim_scale=True,
      use_rotary_position_embeddings=config.use_rotary_position_embeddings,
      qk_norm=config.qk_norm,
      rngs=rngs,
    )

    if config.feedforward_norm == "rms":
      self.pre_ff_ln = RMSNorm(num_features=config.model_dims, rngs=rngs)
      self.post_ff_ln = RMSNorm(num_features=config.model_dims, rngs=rngs)
    else:
      raise ValueError(f"Layer norm: {config.feedforward_norm} not supported.")
    self.ff0 = nnx.Linear(
      in_features=config.model_dims,
      out_features=config.hidden_dims,

View on GitHub (pinned to 331c6d33cb)

Solutions

Set config.attention_norm to 'rms' (exact string, lowercase).
If LayerNorm is needed, add an elif branch constructing nnx.LayerNorm in transformer.py.
Confirm the config source doesn't inject a default like 'layer_norm' when the key is absent.

Example fix

// before
config = TransformerConfig(attention_norm="layer_norm")  # ValueError
// after
config = TransformerConfig(attention_norm="rms")

Defensive patterns

Strategy: validation

Validate before calling

if config.attention_norm != "rms":
    raise ValueError(f"attention_norm must be 'rms', got {config.attention_norm!r}")

Type guard

def has_supported_attention_norm(cfg) -> bool:
    return getattr(cfg, "attention_norm", None) == "rms"

Try / catch

try:
    block = TransformerBlock(config)
except ValueError as e:
    if "attention_norm" in str(e) or "Layer norm" in str(e):
        config.attention_norm = "rms"
        block = TransformerBlock(config)
    else:
        raise

Prevention

Use Literal["rms"] typing (or an enum) for attention_norm in the config class.
Never copy norm names from other frameworks without mapping them first.
Pin the config default to "rms" explicitly.

When it happens

Trigger: Constructing the transformer with config.attention_norm set to 'layer_norm', 'layernorm', 'ln', 'none', or None instead of 'rms'.

Common situations: Porting configs from other transformer stacks that default to LayerNorm, hand-editing configs expecting more norm options, or typos like 'RMS' (case-sensitive comparison).

Related errors

AI-assisted analysis of google-research/timesfm@331c6d33cb (2026-08-29). Data as JSON: /api/errors/ab214fb8fe3a11cb. Report an issue: GitHub.