{"record":{"id":"7341f8ac2282f974","repo":"keras-team/keras","slug":"cannot-set-an-empty-vocabulary-received-vocabula","errorCode":null,"errorMessage":"Cannot set an empty vocabulary. Received: vocabulary={vocabulary}","messagePattern":"Cannot set an empty vocabulary\\. Received: vocabulary=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"keras/src/layers/preprocessing/index_lookup.py","lineNumber":493,"sourceCode":"                \"when not executing eagerly. \"\n                \"Create this layer or call `set_vocabulary()` \"\n                \"outside of any traced function.\"\n            )\n\n        # TODO(mattdangerw): for better performance we should rewrite this\n        # entire function to operate on tensors and convert vocabulary to a\n        # tensor here.\n        if tf.is_tensor(vocabulary):\n            vocabulary = self._tensor_vocab_to_numpy(vocabulary)\n        elif isinstance(vocabulary, (list, tuple)):\n            vocabulary = np.array(vocabulary)\n        if tf.is_tensor(idf_weights):\n            idf_weights = idf_weights.numpy()\n        elif isinstance(idf_weights, (list, tuple)):\n            idf_weights = np.array(idf_weights)\n\n        if vocabulary.size == 0:\n            raise ValueError(\n                \"Cannot set an empty vocabulary. \"\n                f\"Received: vocabulary={vocabulary}\"\n            )\n\n        oov_start = self._oov_start_index()\n        token_start = self._token_start_index()\n        special_tokens = [self.mask_token] * oov_start + [\n            self.oov_token\n        ] * self.num_oov_indices\n        found_special_tokens = np.array_equal(\n            special_tokens, vocabulary[:token_start]\n        )\n        if found_special_tokens:\n            tokens = vocabulary[token_start:]\n        else:\n            tokens = vocabulary\n\n        repeated_tokens = self._find_repeated_tokens(tokens)","sourceCodeStart":475,"sourceCodeEnd":511,"githubUrl":"https://github.com/keras-team/keras/blob/7a34a03db60bf60042242d6a556fc3be119046a5/keras/src/layers/preprocessing/index_lookup.py#L475-L511","documentation":"An empty vocabulary would leave the lookup table with zero entries and break every downstream index computation, so set_vocabulary rejects arrays of size 0 instead of silently continuing.","triggerScenarios":"`set_vocabulary(np.array([]))` or an empty list; adapt() on a dataset slice that yields no tokens; parsing a vocabulary file to zero lines.","commonSituations":"Adapting on a fully filtered-out dataset; empty vocabulary files; conditional data paths that occasionally produce no rows.","solutions":["Check `len(vocab) > 0` before calling set_vocabulary.","If the vocabulary came from adapt, inspect the input dataset — it is yielding nothing."],"exampleFix":"# before\nlayer.set_vocabulary(vocab)\n\n# after\nassert len(vocab) > 0, 'vocabulary is empty'\nlayer.set_vocabulary(vocab)","handlingStrategy":"validation","validationCode":"if len(vocab) == 0:\n    raise ValueError('refusing to set an empty vocabulary')\nlayer.set_vocabulary(vocab)","typeGuard":"def has_tokens(v) -> bool:\n    return len(v) > 0","tryCatchPattern":null,"preventionTips":["Assert non-empty vocabularies right after generation or adapt.","Log dataset sizes before adapting so empty slices are visible early."],"tags":["keras","preprocessing","index-lookup","vocabulary","empty-input"],"backgroundTag":"empty-input","analyzedSha":"7a34a03db60bf60042242d6a556fc3be119046a5","analyzedAt":"2026-08-25T21:25:25.994Z","schemaVersion":2},"datasetVersion":"2026-08-26T02:17:13.382Z"}