tensorflow/models · error · ValueError

Setting `diff_q_and_kv_attention_layer_norm` to Truewhen `no

Error message

Setting `diff_q_and_kv_attention_layer_norm` to Truewhen `norm_first` is False is invalid.

What it means

Error "Setting `diff_q_and_kv_attention_layer_norm` to Truewhen `norm_first` is False is invalid." thrown in tensorflow/models.

Source

Thrown at official/nlp/modeling/layers/transformer_encoder_block.py:280

        and self._num_kv_heads is not None
        and self._num_kv_heads != 1
    ):
      raise ValueError(
          "Block sparse attention only supports Multi-query attention.Please"
          " set num_kv_heads to 1 to enable MQA with block sparse attention."
      )
    if attention_initializer:
      self._attention_initializer = tf_keras.initializers.get(
          attention_initializer
      )
    else:
      self._attention_initializer = tf_utils.clone_initializer(
          self._kernel_initializer
      )
    self._attention_axes = attention_axes

    if self._diff_q_kv_att_layer_norm and not self._norm_first:
      raise ValueError(
          "Setting `diff_q_and_kv_attention_layer_norm` to True"
          "when `norm_first` is False is invalid."
      )

  def build(self, input_shape):
    if isinstance(input_shape, tf.TensorShape):
      input_tensor_shape = input_shape
    elif isinstance(input_shape, (list, tuple)):
      input_tensor_shape = tf.TensorShape(input_shape[0])
    elif isinstance(input_shape, dict):
      input_tensor_shape = tf.TensorShape(input_shape["input_tensor"])
    else:
      raise ValueError(
          "The type of input shape argument is not supported, got: %s" %
          type(input_shape))
    einsum_equation = "abc,cd->abd"
    if len(input_tensor_shape.as_list()) > 3:
      einsum_equation = "...bc,cd->...bd"

View on GitHub (pinned to e006f5f0d5)

When it happens

Trigger: Thrown at official/nlp/modeling/layers/transformer_encoder_block.py:280 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of tensorflow/models@e006f5f0d5 (2026-08-24). Data as JSON: /api/errors/d325dac106d9fdaf. Report an issue: GitHub.