tensorflow/models · error · ValueError

`seq_length` should be a multiple of `permutation_size`.

Error message

`seq_length` should be a multiple of `permutation_size`.

What it means

Error "`seq_length` should be a multiple of `permutation_size`." thrown in tensorflow/models.

Source

Thrown at official/nlp/data/pretrain_dataloader.py:283

      perm_mask_0 = tf.concat([
          perm_mask_0,
          tf.zeros([self._reuse_length, non_reuse_len], dtype=tf.int32)
      ],
                              axis=1)
      perm_mask_1 = tf.concat([
          tf.ones([non_reuse_len, self._reuse_length], dtype=tf.int32),
          perm_mask_1
      ],
                              axis=1)
      perm_mask = tf.concat([perm_mask_0, perm_mask_1], axis=0)
      target_mask = tf.concat([target_mask_0, target_mask_1], axis=0)
      tokens = tf.concat([tokens_0, tokens_1], axis=0)
      masked_tokens = tf.concat([masked_0, masked_1], axis=0)
    else:
      # Disable the memory mechanism.
      if self._seq_length % self._permutation_size != 0:
        raise ValueError('`seq_length` should be a multiple of '
                         '`permutation_size`.')
      # Permute the entire sequence together
      perm_mask, target_mask, tokens, masked_tokens = self._get_factorization(
          inputs=inputs, input_mask=input_mask)
    x['permutation_mask'] = tf.reshape(perm_mask,
                                       [self._seq_length, self._seq_length])
    x['input_word_ids'] = tokens
    x['masked_tokens'] = masked_tokens

    target = tokens
    if self._max_predictions_per_seq is not None:
      indices = tf.range(self._seq_length, dtype=tf.int32)
      bool_target_mask = tf.cast(target_mask, tf.bool)
      indices = tf.boolean_mask(indices, bool_target_mask)

      # account for extra padding due to CLS/SEP.
      actual_num_predict = tf.shape(indices)[0]
      pad_len = self._max_predictions_per_seq - actual_num_predict

View on GitHub (pinned to e006f5f0d5)

When it happens

Trigger: Thrown at official/nlp/data/pretrain_dataloader.py:283 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of tensorflow/models@e006f5f0d5 (2026-08-24). Data as JSON: /api/errors/6cb4bd5bdb5cdeac. Report an issue: GitHub.