tensorflow/models · error · ValueError

`masked_tokens` must be provided in order to initialize the

Error message

`masked_tokens` must be provided in order to initialize the query stream in `TwoStreamRelativeAttention`.

What it means

Error "`masked_tokens` must be provided in order to initialize the query stream in `TwoStreamRelativeAttention`." thrown in tensorflow/models.

Source

Thrown at official/nlp/modeling/networks/xlnet_base.py:625

    """Implements call() for the layer."""
    input_ids = inputs["input_ids"]
    segment_ids = inputs["segment_ids"]
    input_mask = inputs["input_mask"]
    state = inputs["state"]
    permutation_mask = inputs["permutation_mask"]
    target_mapping = inputs["target_mapping"]
    masked_tokens = inputs["masked_tokens"]

    batch_size = tf.shape(input_ids)[0]
    seq_length = tf.shape(input_ids)[1]
    if state is not None:
      memory_length = tf.shape(state[0])[1]
    else:
      memory_length = 0
    total_length = memory_length + seq_length

    if self._two_stream and masked_tokens is None:
      raise ValueError("`masked_tokens` must be provided in order to "
                       "initialize the query stream in "
                       "`TwoStreamRelativeAttention`.")
    if masked_tokens is not None and not self._two_stream:
      logging.warning("`masked_tokens` is provided but `two_stream` is not "
                      "enabled. Please enable `two_stream` to enable two "
                      "stream attention.")

    if input_mask is not None:
      dtype = input_mask.dtype
    elif permutation_mask is not None:
      dtype = permutation_mask.dtype
    else:
      dtype = tf.int32
    query_attention_mask, content_attention_mask = _compute_attention_mask(
        input_mask=input_mask,
        permutation_mask=permutation_mask,
        attention_type=self._attention_type,
        seq_length=seq_length,

View on GitHub (pinned to e006f5f0d5)

When it happens

Trigger: Thrown at official/nlp/modeling/networks/xlnet_base.py:625 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of tensorflow/models@e006f5f0d5 (2026-08-24). Data as JSON: /api/errors/ff48c9ec370be3ac. Report an issue: GitHub.