{"record":{"id":"79365ad00c0113e3","repo":"google-research/timesfm","slug":"activation-config-ff-activation-not-supported-79365a","errorCode":null,"errorMessage":"Activation: {config.ff_activation} not supported.","messagePattern":"Activation: (.+?) not supported\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/timesfm/torch/transformer.py","lineNumber":352,"sourceCode":"\n    self.ff0 = nn.Linear(\n      in_features=config.model_dims,\n      out_features=config.hidden_dims,\n      bias=config.use_bias,\n    )\n    self.ff1 = nn.Linear(\n      in_features=config.hidden_dims,\n      out_features=config.model_dims,\n      bias=config.use_bias,\n    )\n    if config.ff_activation == \"relu\":\n      self.activation = nn.ReLU()\n    elif config.ff_activation == \"swish\":\n      self.activation = nn.SiLU()\n    elif config.ff_activation == \"none\":\n      self.activation = nn.Identity()\n    else:\n      raise ValueError(f\"Activation: {config.ff_activation} not supported.\")\n\n  def forward(\n    self,\n    input_embeddings: torch.Tensor,\n    patch_mask: torch.Tensor,\n    decode_cache: DecodeCache | None = None,\n  ) -> tuple[torch.Tensor, DecodeCache | None]:\n    attn_output, decode_cache = self.attn(\n      inputs_q=self.pre_attn_ln(input_embeddings),\n      decode_cache=decode_cache,\n      patch_mask=patch_mask,\n    )\n    attn_output = self.post_attn_ln(attn_output) + input_embeddings\n    output_embeddings = (\n      self.post_ff_ln(self.ff1(self.activation(self.ff0(self.pre_ff_ln(attn_output)))))\n      + attn_output\n    )\n    return output_embeddings, decode_cache","sourceCodeStart":334,"sourceCodeEnd":370,"githubUrl":"https://github.com/google-research/timesfm/blob/331c6d33cb1ac2611de3056d0ac7164aab6301eb/src/timesfm/torch/transformer.py#L334-L370","documentation":"The Torch transformer's feedforward activation is selected in __init__ from a fixed set: \"relu\", \"swish\", \"none\" (plus whatever the preceding branch, likely \"gelu\", handles). Any other config.ff_activation value raises this ValueError because no nn activation module is assigned.","triggerScenarios":"Creating the transformer block with config.ff_activation set to an unsupported string, e.g. \"silu\", \"swish/\", \"GELU\", or None.","commonSituations":"Typos or wrong casing when hand-writing configs; using names from other frameworks (e.g. \"silu\" instead of \"swish\"); migrating configs between JAX and Torch TimesFM implementations with differing enum sets.","solutions":["Set config.ff_activation to one of the supported values: \"relu\", \"swish\", \"none\" (or \"gelu\" if supported by the preceding branch).","Use \"swish\" instead of \"silu\" — nn.SiLU is what \"swish\" maps to.","Fix casing; the comparison is exact lowercase string matching."],"exampleFix":"// before\nconfig.ff_activation = \"silu\"\n// after\nconfig.ff_activation = \"swish\"","handlingStrategy":"validation","validationCode":"ALLOWED = {\"relu\", \"swish\", \"none\", \"gelu\"}\nif config.ff_activation not in ALLOWED:\n    raise ValueError(f\"ff_activation must be one of {ALLOWED}, got {config.ff_activation!r}\")","typeGuard":"def has_valid_activation(config) -> bool:\n    return getattr(config, \"ff_activation\", None) in {\"relu\", \"swish\", \"none\", \"gelu\"}","tryCatchPattern":"try:\n    model = TimesFmTorch(config)\nexcept ValueError as e:\n    if \"Activation\" in str(e):\n        config.ff_activation = \"swish\"\n        model = TimesFmTorch(config)\n    else:\n        raise","preventionTips":["Use \"swish\" not \"silu\" for this library","Keep activation names lowercase","Validate the whole config once at startup instead of at each model build"],"tags":["python","configuration","valueerror"],"backgroundTag":"unsupported-config-value","analyzedSha":"331c6d33cb1ac2611de3056d0ac7164aab6301eb","analyzedAt":"2026-08-29T01:04:23.138Z","schemaVersion":2},"datasetVersion":"2026-08-29T02:17:18.158Z"}