google-research/timesfm · error · ValueError

Activation: {config.ff_activation} not supported.

Error message

Activation: {config.ff_activation} not supported.

What it means

The Torch transformer's feedforward activation is selected in __init__ from a fixed set: "relu", "swish", "none" (plus whatever the preceding branch, likely "gelu", handles). Any other config.ff_activation value raises this ValueError because no nn activation module is assigned.

Source

Thrown at src/timesfm/torch/transformer.py:352

    self.ff0 = nn.Linear(
      in_features=config.model_dims,
      out_features=config.hidden_dims,
      bias=config.use_bias,
    )
    self.ff1 = nn.Linear(
      in_features=config.hidden_dims,
      out_features=config.model_dims,
      bias=config.use_bias,
    )
    if config.ff_activation == "relu":
      self.activation = nn.ReLU()
    elif config.ff_activation == "swish":
      self.activation = nn.SiLU()
    elif config.ff_activation == "none":
      self.activation = nn.Identity()
    else:
      raise ValueError(f"Activation: {config.ff_activation} not supported.")

  def forward(
    self,
    input_embeddings: torch.Tensor,
    patch_mask: torch.Tensor,
    decode_cache: DecodeCache | None = None,
  ) -> tuple[torch.Tensor, DecodeCache | None]:
    attn_output, decode_cache = self.attn(
      inputs_q=self.pre_attn_ln(input_embeddings),
      decode_cache=decode_cache,
      patch_mask=patch_mask,
    )
    attn_output = self.post_attn_ln(attn_output) + input_embeddings
    output_embeddings = (
      self.post_ff_ln(self.ff1(self.activation(self.ff0(self.pre_ff_ln(attn_output)))))
      + attn_output
    )
    return output_embeddings, decode_cache

View on GitHub (pinned to 331c6d33cb)

Solutions

1. Set config.ff_activation to one of the supported values: "relu", "swish", "none" (or "gelu" if supported by the preceding branch).
2. Use "swish" instead of "silu" — nn.SiLU is what "swish" maps to.
3. Fix casing; the comparison is exact lowercase string matching.

Example fix

// before
config.ff_activation = "silu"
// after
config.ff_activation = "swish"

ALLOWED = {"relu", "swish", "none", "gelu"}
if config.ff_activation not in ALLOWED:
    raise ValueError(f"ff_activation must be one of {ALLOWED}, got {config.ff_activation!r}")

def has_valid_activation(config) -> bool:
    return getattr(config, "ff_activation", None) in {"relu", "swish", "none", "gelu"}

try:
    model = TimesFmTorch(config)
except ValueError as e:
    if "Activation" in str(e):
        config.ff_activation = "swish"
        model = TimesFmTorch(config)
    else:
        raise

When it happens

Trigger: Creating the transformer block with config.ff_activation set to an unsupported string, e.g. "silu", "swish/", "GELU", or None.

Common situations: Typos or wrong casing when hand-writing configs; using names from other frameworks (e.g. "silu" instead of "swish"); migrating configs between JAX and Torch TimesFM implementations with differing enum sets.

Related errors

• Layer norm: {config.attention_norm} not supported.
• Layer norm: {config.feedforward_norm} not supported.
• Unsupported array shape: {x.shape}
• train_dynamic_numerical_covariates and test_dynamic_numerica
• train_dynamic_categorical_covariates and test_dynamic_catego

AI-assisted analysis of google-research/timesfm@331c6d33cb (2026-08-29). Data as JSON: /api/errors/79365ad00c0113e3. Report an issue: GitHub.