tensorflow/models · error · ValueError

window_decay should be in (0.0, 1.0) and not None.

Error message

window_decay should be in (0.0, 1.0) and not None.

What it means

Error "window_decay should be in (0.0, 1.0) and not None." thrown in tensorflow/models.

Source

Thrown at official/nlp/modeling/layers/kernel_attention.py:262

      k_winsum = weighted_window_sum(k_sum, window_length, decaying_weights)

    numerator = tf.einsum(
        "BTCHD,BTHDO->BTCHO", chunked_query_matrix, kp_v_winsum)

    k_winsum = tf.squeeze(k_winsum, -3)
    denominator = tf.einsum("BTCHD,BTHD->BTCH", chunked_query_matrix, k_winsum)
    denominator = tf.expand_dims(denominator, -1) + _NUMERIC_STABLER
    attention = numerator / denominator
    attention = tf.reshape(attention, new_shape)

    start = tf.zeros([old_shape.shape[0]], dtype=old_shape.dtype)
    attention = tf.slice(attention, start, old_shape)

  # Queued window cache (drop instead of decay) not yet supported.
  else:  # Streaming

    if window_decay is None or window_decay > 1.0 or window_decay < 0.0:
      raise ValueError("window_decay should be in (0.0, 1.0) and not None.")
    kv = window_decay * cache["kv"] + tf.einsum(
        "BTHD,BTHO->BHOD", key_matrix, value_matrix)
    cache["kv"] = kv
    k_sum = window_decay * cache["k_sum"] + tf.reduce_sum(key_matrix, axis=1)
    cache["k_sum"] = k_sum
    denominator = tf.einsum("BTHD,BHD->BTH", query_matrix, k_sum)
    # The below is equivalent to but converts to TF Lite better than:
    #   tf.einsum("BTHD,BTH->BTHD",
    #             query_matrix, 1.0 / (denominator + _NUMERIC_STABLER))
    inverse_denominator = 1.0 / (denominator + _NUMERIC_STABLER)
    # Add another dimension to align for the broadcast multiplication.
    fused_query_denominator = query_matrix * tf.expand_dims(inverse_denominator,
                                                            -1)
    attention = tf.einsum("BTHD,BHOD->BTHO", fused_query_denominator, kv)
  return attention


def create_projection_matrix(m, d, seed=None):

View on GitHub (pinned to e006f5f0d5)

When it happens

Trigger: Thrown at official/nlp/modeling/layers/kernel_attention.py:262 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of tensorflow/models@e006f5f0d5 (2026-08-24). Data as JSON: /api/errors/2c83e29329a2f4bd. Report an issue: GitHub.