{"record":{"id":"86e5f88ce6de9856","repo":"tensorflow/models","slug":"cannot-use-ngram-masking-without-whole-word-maskin","errorCode":null,"errorMessage":"cannot use ngram masking without whole word masking","messagePattern":"cannot use ngram masking without whole word masking","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"official/nlp/data/create_pretraining_data.py","lineNumber":597,"sourceCode":"      gram_start_pos = i\n    else:\n      gram_start_pos = None\n  if gram_start_pos is not None:\n    grams.append(_Gram(gram_start_pos, len(tokens)))\n  return grams\n\n\ndef create_masked_lm_predictions(tokens, masked_lm_prob,\n                                 max_predictions_per_seq, vocab_words, rng,\n                                 do_whole_word_mask,\n                                 max_ngram_size=None):\n  \"\"\"Creates the predictions for the masked LM objective.\"\"\"\n  if do_whole_word_mask:\n    grams = _tokens_to_grams(tokens)\n  else:\n    # Here we consider each token to be a word to allow for sub-word masking.\n    if max_ngram_size:\n      raise ValueError(\"cannot use ngram masking without whole word masking\")\n    grams = [_Gram(i, i+1) for i in range(0, len(tokens))\n             if tokens[i] not in [\"[CLS]\", \"[SEP]\"]]\n\n  num_to_predict = min(max_predictions_per_seq,\n                       max(1, int(round(len(tokens) * masked_lm_prob))))\n  # Generate masks.  If `max_ngram_size` in [0, None] it means we're doing\n  # whole word masking or token level masking.  Both of these can be treated\n  # as the `max_ngram_size=1` case.\n  masked_grams = _masking_ngrams(grams, max_ngram_size or 1,\n                                 num_to_predict, rng)\n  masked_lms = []\n  output_tokens = list(tokens)\n  for gram in masked_grams:\n    # 80% of the time, replace all n-gram tokens with [MASK]\n    if rng.random() < 0.8:\n      replacement_action = lambda idx: \"[MASK]\"\n    else:\n      # 10% of the time, keep all the original n-gram tokens.","sourceCodeStart":579,"sourceCodeEnd":615,"githubUrl":"https://github.com/tensorflow/models/blob/e006f5f0d534913e49c1f1dae87364039fa607e2/official/nlp/data/create_pretraining_data.py#L579-L615","documentation":"Error \"cannot use ngram masking without whole word masking\" thrown in tensorflow/models.","triggerScenarios":"Thrown at official/nlp/data/create_pretraining_data.py:597 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":[],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"e006f5f0d534913e49c1f1dae87364039fa607e2","analyzedAt":"2026-08-24T14:09:15.576Z","schemaVersion":2},"datasetVersion":"2026-08-24T17:17:21.512Z"}