{"record":{"id":"b56000a3076f2222","repo":"keras-team/keras","slug":"found-reserved-oov-token-at-unexpected-location-in","errorCode":null,"errorMessage":"Found reserved OOV token at unexpected location in `vocabulary`. Note that passed `vocabulary` does not need to include the OOV and mask tokens. Either remove all mask and OOV tokens, or include them only at the start of the vocabulary in precisely this order: {special_tokens}. Received: oov_token={self.oov_token} at vocabulary index {oov_index}","messagePattern":"Found reserved OOV token at unexpected location in `vocabulary`\\. Note that passed `vocabulary` does not need to include the OOV and mask tokens\\. Either remove all mask and OOV tokens, or include them only at the start of the vocabulary in precisely this order: (.+?)\\. Received: oov_token=(.+?) at vocabulary index (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"keras/src/layers/preprocessing/index_lookup.py","lineNumber":538,"sourceCode":"            mask_index = np.argwhere(vocabulary == self.mask_token)[-1]\n            raise ValueError(\n                \"Found reserved mask token at unexpected location in \"\n                \"`vocabulary`. Note that passed `vocabulary` does not need to \"\n                \"include the OOV and mask tokens. Either remove all mask and \"\n                \"OOV tokens, or include them only at the start of the \"\n                f\"vocabulary in precisely this order: {special_tokens}. \"\n                f\"Received: mask_token={self.mask_token} at \"\n                f\"vocabulary index {mask_index}\"\n            )\n        # Only error out for oov_token when invert=True. When invert=False,\n        # oov_token is unused during lookup.\n        if (\n            self.oov_token is not None\n            and self.invert\n            and self.oov_token in tokens\n        ):\n            oov_index = np.argwhere(vocabulary == self.oov_token)[-1]\n            raise ValueError(\n                \"Found reserved OOV token at unexpected location in \"\n                \"`vocabulary`. Note that passed `vocabulary` does not need to \"\n                \"include the OOV and mask tokens. Either remove all mask and \"\n                \"OOV tokens, or include them only at the start of the \"\n                f\"vocabulary in precisely this order: {special_tokens}. \"\n                f\"Received: oov_token={self.oov_token} at \"\n                f\"vocabulary index {oov_index}\"\n            )\n\n        new_vocab_size = token_start + len(tokens)\n        if self.max_tokens is not None and (new_vocab_size > self.max_tokens):\n            raise ValueError(\n                \"Attempted to set a vocabulary larger than the maximum vocab \"\n                f\"size. Received vocabulary size is {new_vocab_size}; \"\n                f\"`max_tokens` is {self.max_tokens}.\"\n            )\n        self.lookup_table = self._lookup_table_from_tokens(tokens)\n        self._record_vocabulary_size()","sourceCodeStart":520,"sourceCodeEnd":556,"githubUrl":"https://github.com/keras-team/keras/blob/7a34a03db60bf60042242d6a556fc3be119046a5/keras/src/layers/preprocessing/index_lookup.py#L520-L556","documentation":"The OOV-token counterpart of the mask-token check, applied on inverted (index-to-token) layers: if the oov_token appears in the passed vocabulary outside its reserved head slot, the reverse mapping would be ambiguous.","triggerScenarios":"`layer.set_vocabulary(vocab)` on a layer with `invert=True` where vocab contains '[UNK]' at any position other than the reserved slot.","commonSituations":"Building a decoder lookup from a vocabulary file that already includes the '[UNK]' token.","solutions":["Strip the oov token from the vocabulary before passing it.","Or place it exactly at the reserved head position per the documented special-token order.","Ensure encoder and decoder layers use consistent oov_token and num_oov_indices settings."],"exampleFix":"# before\ninv_layer.set_vocabulary(vocab)  # vocab includes '[UNK]'\n\n# after\ntokens = [t for t in vocab if t != '[UNK]']\ninv_layer.set_vocabulary(tokens)","handlingStrategy":"validation","validationCode":"vocab = [t for t in vocab if t != '[UNK]']\ninv_layer.set_vocabulary(vocab)","typeGuard":"def free_of_oov(vocab, oov_token) -> bool:\n    return oov_token not in vocab","tryCatchPattern":null,"preventionTips":["Strip the oov token from decoder vocabularies; the layer reserves its own slot.","Keep encoder and decoder special-token settings identical."],"tags":["keras","preprocessing","index-lookup","vocabulary","special-tokens"],"backgroundTag":"reserved-token-misplaced","analyzedSha":"7a34a03db60bf60042242d6a556fc3be119046a5","analyzedAt":"2026-08-25T21:25:25.994Z","schemaVersion":2},"datasetVersion":"2026-08-26T02:17:13.382Z"}