{"record":{"id":"a56c977c16f51a5e","repo":"google-research/timesfm","slug":"activation-config-ff-activation-not-supported","errorCode":null,"errorMessage":"Activation: {config.ff_activation} not supported.","messagePattern":"Activation: (.+?) not supported\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/timesfm/flax/transformer.py","lineNumber":336,"sourceCode":"      in_features=config.model_dims,\n      out_features=config.hidden_dims,\n      use_bias=config.use_bias,\n      rngs=rngs,\n    )\n    self.ff1 = nnx.Linear(\n      in_features=config.hidden_dims,\n      out_features=config.model_dims,\n      use_bias=config.use_bias,\n      rngs=rngs,\n    )\n    if config.ff_activation == \"relu\":\n      self.activation = jax.nn.relu\n    elif config.ff_activation == \"swish\":\n      self.activation = jax.nn.swish\n    elif config.ff_activation == \"none\":\n      self.activation = lambda x: x\n    else:\n      raise ValueError(f\"Activation: {config.ff_activation} not supported.\")\n\n  def __call__(\n    self,\n    input_embeddings: Float[Array, \"b n d\"],\n    patch_mask: Bool[Array, \"b n\"],\n    decode_cache: DecodeCache | None = None,\n  ) -> tuple[Float[Array, \"b n d\"], DecodeCache | None]:\n    attn_output, decode_cache = self.attn(\n      inputs_q=self.pre_attn_ln(input_embeddings),\n      decode_cache=decode_cache,\n      patch_mask=patch_mask,\n      sow_weights=False,\n      deterministic=True,\n    )\n    attn_output = self.post_attn_ln(attn_output) + input_embeddings\n    output_embeddings = (\n      self.post_ff_ln(self.ff1(self.activation(self.ff0(self.pre_ff_ln(attn_output)))))\n      + attn_output","sourceCodeStart":318,"sourceCodeEnd":354,"githubUrl":"https://github.com/google-research/timesfm/blob/331c6d33cb1ac2611de3056d0ac7164aab6301eb/src/timesfm/flax/transformer.py#L318-L354","documentation":"The feed-forward network only supports activation names 'relu', 'swish', and 'none'; any other config.ff_activation value falls through to this ValueError in __init__. Same pattern as the dense-layer activation check but keyed on ff_activation.","triggerScenarios":"Constructing the transformer with config.ff_activation set to 'gelu', 'tanh', 'elu', 'gelu_new', or None instead of 'relu', 'swish', or 'none'.","commonSituations":"Porting configs from HF/other frameworks whose default FF activation is 'gelu'; casing mistakes ('ReLU'); missing field causing a None default.","solutions":["Set config.ff_activation to 'relu', 'swish', or 'none'.","To use another activation, add an elif branch mapping it to the jax.nn function in transformer.py.","Check the config file/default dict for the injected value when the key is omitted."],"exampleFix":"// before\nconfig = TransformerConfig(ff_activation=\"gelu\")  # ValueError\n// after\nconfig = TransformerConfig(ff_activation=\"swish\")","handlingStrategy":"validation","validationCode":"SUPPORTED_FF_ACT = {\"relu\", \"swish\", \"none\"}\nif config.ff_activation not in SUPPORTED_FF_ACT:\n    raise ValueError(f\"ff_activation must be one of {sorted(SUPPORTED_FF_ACT)}, got {config.ff_activation!r}\")","typeGuard":"def is_supported_ff_activation(a: object) -> bool:\n    return isinstance(a, str) and a in {\"relu\", \"swish\", \"none\"}","tryCatchPattern":"try:\n    block = TransformerBlock(config)\nexcept ValueError as e:\n    if \"ff_activation\" in str(e) or \"Activation\" in str(e):\n        config.ff_activation = \"swish\"\n        block = TransformerBlock(config)\n    else:\n        raise","preventionTips":["Map foreign config activations (e.g. gelu -> swish) at config import time.","Use Literal[\"relu\",\"swish\",\"none\"] typing for ff_activation.","Construct the block in CI tests with each supported value."],"tags":["config","validation","activation","valueerror"],"backgroundTag":"unsupported-config-value","analyzedSha":"331c6d33cb1ac2611de3056d0ac7164aab6301eb","analyzedAt":"2026-08-29T01:04:23.138Z","schemaVersion":2},"datasetVersion":"2026-08-29T02:17:18.158Z"}