tensorflow/models · error · ValueError

{self.__class__} does not support block sparse attention.

Error message

{self.__class__} does not support block sparse attention.

What it means

Error "{self.__class__} does not support block sparse attention." thrown in tensorflow/models.

Source

Thrown at official/nlp/modeling/layers/pack_optimization.py:146

    if self._norm_first:
      return source_attention_output + layer_output  # pyrefly: ignore[unbound-name]

    layer_output = tf.cast(layer_output, tf.float32)
    return self._output_layer_norm(layer_output + attention_output)


@tf_keras.utils.register_keras_serializable(package='Text')
class StridedReZeroTransformer(rezero_transformer.ReZeroTransformer):
  """ReZeroTransformer for packing optimization to stride over inputs."""

  def __init__(self, *args, **kwargs):
    super().__init__(*args, **kwargs)
    if self._output_range is not None:
      raise ValueError(f'{self.__class__} does not '
                       'support `output_range` argument.')
    # TODO(b/337888023): Support block sparse attention with strided inputs.
    if self._src_block_size is not None:
      raise ValueError(f'{self.__class__} does not '
                       'support block sparse attention.')

  def call(self, inputs, stride: tf.Tensor):
    if isinstance(inputs, (list, tuple)):
      if len(inputs) == 2:
        input_tensor, attention_mask = inputs
        key_value = None
      elif len(inputs) == 3:
        input_tensor, key_value, attention_mask = inputs
      else:
        raise ValueError(f'Unexpected inputs to {self.__class__} with '
                         f'length at {len(inputs)}.')
    else:
      input_tensor, key_value, attention_mask = (inputs, None, None)

    target_tensor = input_tensor[:, ::stride, :]
    if attention_mask is not None:
      attention_mask = attention_mask[:, ::stride, :]

View on GitHub (pinned to e006f5f0d5)

When it happens

Trigger: Thrown at official/nlp/modeling/layers/pack_optimization.py:146 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of tensorflow/models@e006f5f0d5 (2026-08-24). Data as JSON: /api/errors/b70f28ce64789599. Report an issue: GitHub.