tensorflow/models · error · ValueError

Setting `diff_q_and_kv_attention_layer_norm` to Truewhen `no

Error message

Setting `diff_q_and_kv_attention_layer_norm` to Truewhen `norm_first` is False is invalid.

What it means

Error "Setting `diff_q_and_kv_attention_layer_norm` to Truewhen `norm_first` is False is invalid." thrown in tensorflow/models.

Source

Thrown at official/projects/lra/linformer_encoder_block.py:173

    self._inner_dropout = inner_dropout
    self._use_query_residual = use_query_residual
    self._key_dim = key_dim
    self._value_dim = value_dim
    self._output_last_dim = output_last_dim
    self._diff_q_kv_att_layer_norm = diff_q_kv_att_layer_norm
    self._return_attention_scores = return_attention_scores
    if attention_initializer:
      self._attention_initializer = tf_keras.initializers.get(
          attention_initializer
      )
    else:
      self._attention_initializer = tf_utils.clone_initializer(
          self._kernel_initializer
      )
    self._attention_axes = attention_axes

    if self._diff_q_kv_att_layer_norm and not self._norm_first:
      raise ValueError(
          "Setting `diff_q_and_kv_attention_layer_norm` to True"
          "when `norm_first` is False is invalid."
      )

  def build(self, input_shape):
    if isinstance(input_shape, tf.TensorShape):
      input_tensor_shape = input_shape
    elif isinstance(input_shape, (list, tuple)):
      input_tensor_shape = tf.TensorShape(input_shape[0])
    else:
      raise ValueError(
          "The type of input shape argument is not supported, got: %s"
          % type(input_shape)
      )
    einsum_equation = "abc,cd->abd"
    if len(input_tensor_shape.as_list()) > 3:
      einsum_equation = "...bc,cd->...bd"
    hidden_size = input_tensor_shape[-1]

View on GitHub (pinned to e006f5f0d5)

When it happens

Trigger: Thrown at official/projects/lra/linformer_encoder_block.py:173 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of tensorflow/models@e006f5f0d5 (2026-08-24). Data as JSON: /api/errors/88aca33f46fa729f. Report an issue: GitHub.