{"record":{"id":"ca2c5eb02d72675e","repo":"tensorflow/models","slug":"in-the-case-of-norm-first-the-residual-connecti","errorCode":null,"errorMessage":"In the case of `norm_first`, the residual connection should be done in the TransformerScaffold call function, not FFN's call function.","messagePattern":"In the case of `norm_first`, the residual connection should be done in the TransformerScaffold call function, not FFN's call function\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"official/vision/modeling/layers/nn_blocks.py","lineNumber":2843,"sourceCode":"          source_attention_output)\n    else:\n      attention_output = self._attention_layer_norm(\n          input_tensor +\n          self._stochastic_depth(attention_output, training=training))\n\n    if self._feedforward_block is None:\n      intermediate_output = self._intermediate_dense(attention_output)\n      intermediate_output = self._intermediate_activation_layer(\n          intermediate_output)\n      layer_output = self._output_dense(intermediate_output)\n      layer_output = self._output_dropout(layer_output, training=training)\n    else:\n      layer_output = self._feedforward_block(\n          attention_output, training=training)\n\n    if self._norm_first:\n      if self._ffn_has_residual_connection:\n        raise ValueError(\n            'In the case of `norm_first`, the residual connection should be'\n            \"done in the TransformerScaffold call function, not FFN's\"\n            'call function.')\n      output = source_attention_output + self._stochastic_depth(  # pyrefly: ignore[unbound-name]\n          layer_output, training=training)\n    else:\n      # During mixed precision training, layer norm output is always fp32 for\n      # now. Casts fp32 for the subsequent add.\n      layer_output = tf.cast(layer_output, tf.float32)\n      if self._ffn_has_residual_connection:\n        output = self._stochastic_depth(layer_output, training=training)\n      else:\n        output = self._output_layer_norm(\n            attention_output +\n            self._stochastic_depth(layer_output, training=training))\n\n    if self._return_attention_scores:\n      return output, attention_scores","sourceCodeStart":2825,"sourceCodeEnd":2861,"githubUrl":"https://github.com/tensorflow/models/blob/e006f5f0d534913e49c1f1dae87364039fa607e2/official/vision/modeling/layers/nn_blocks.py#L2825-L2861","documentation":"Error \"In the case of `norm_first`, the residual connection should be done in the TransformerScaffold call function, not FFN's call function.\" thrown in tensorflow/models.","triggerScenarios":"Thrown at official/vision/modeling/layers/nn_blocks.py:2843 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["When norm_first is enabled, remove the residual connection from the FFN's call and let TransformerScaffold handle it.","Set the FFN's inner residual/add options consistently with norm_first."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"e006f5f0d534913e49c1f1dae87364039fa607e2","analyzedAt":"2026-08-24T14:09:15.576Z","schemaVersion":2},"datasetVersion":"2026-08-24T17:17:21.512Z"}