{"record":{"id":"6f0babc2ead6a2a9","repo":"keras-team/keras","slug":"when-using-output-mode-self-output-mode-and-p-6f0bab","errorCode":null,"errorMessage":"When using `output_mode={self.output_mode}` and `pad_to_max_tokens=False`, the vocabulary size cannot be changed after the layer is called. Old vocab size is {self._frozen_vocab_size}, new vocab size is {new_vocab_size}","messagePattern":"When using `output_mode=(.+?)` and `pad_to_max_tokens=False`, the vocabulary size cannot be changed after the layer is called\\. Old vocab size is (.+?), new vocab size is (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"keras/src/layers/preprocessing/index_lookup.py","lineNumber":1052,"sourceCode":"                f\"When using `output_mode={self.output_mode}` \"\n                \"and `pad_to_max_tokens=False`, \"\n                \"you must set the layer's vocabulary before calling it. Either \"\n                \"pass a `vocabulary` argument to the layer, or call `adapt` \"\n                \"with some sample data.\"\n            )\n\n    def _ensure_vocab_size_unchanged(self):\n        if self.output_mode == \"int\" or self.pad_to_max_tokens:\n            return\n\n        with tf.init_scope():\n            new_vocab_size = self.vocabulary_size()\n\n        if (\n            self._frozen_vocab_size is not None\n            and new_vocab_size != self._frozen_vocab_size\n        ):\n            raise RuntimeError(\n                f\"When using `output_mode={self.output_mode}` \"\n                \"and `pad_to_max_tokens=False`, \"\n                \"the vocabulary size cannot be changed after the layer is \"\n                f\"called. Old vocab size is {self._frozen_vocab_size}, \"\n                f\"new vocab size is {new_vocab_size}\"\n            )\n\n    def _find_repeated_tokens(self, vocabulary):\n        \"\"\"Return all repeated tokens in a vocabulary.\"\"\"\n        vocabulary_set = set(vocabulary)\n        if len(vocabulary) != len(vocabulary_set):\n            return [\n                item\n                for item, count in collections.Counter(vocabulary).items()\n                if count > 1\n            ]\n        else:\n            return []","sourceCodeStart":1034,"sourceCodeEnd":1070,"githubUrl":"https://github.com/keras-team/keras/blob/7a34a03db60bf60042242d6a556fc3be119046a5/keras/src/layers/preprocessing/index_lookup.py#L1034-L1070","documentation":"After a layer with output_mode != 'int' and pad_to_max_tokens=False is first called, its vocabulary size is frozen because downstream output shapes depend on it. Any later operation that changes vocabulary_size() (set_vocabulary, re-adapt, loading new assets) triggers this RuntimeError from _ensure_vocab_size_unchanged.","triggerScenarios":"Calling the layer once, then calling set_vocabulary() with a different-length vocabulary, re-adapting, or deserializing new weights into a live layer; saving a model, then loading and continuing with an altered vocabulary.","commonSituations":"Incremental training that refreshes vocabularies between epochs; serving pipelines that hot-swap vocabulary assets on a warmed-up model.","solutions":["Recreate the layer (and rebuild/recompile the model) when the vocabulary must change","Freeze a fixed output width up front via max_tokens + pad_to_max_tokens=True","Keep the same vocabulary length; only update token->index content, not size"],"exampleFix":"// before\nout = layer(x)\nlayer.set_vocabulary(bigger_vocab)  # RuntimeError\n// after\nlayer = TextVectorization(output_mode='multi_hot', max_tokens=5000, pad_to_max_tokens=True)\nlayer.set_vocabulary(bigger_vocab)","handlingStrategy":"fallback","validationCode":"if layer._frozen_vocab_size is not None and layer.vocabulary_size() != layer._frozen_vocab_size:\n    layer = rebuild_layer_with_new_vocab()  # recreate instead of mutating","typeGuard":null,"tryCatchPattern":"try:\n    layer.set_vocabulary(new_vocab)\nexcept RuntimeError:\n    layer = build_fresh_layer(new_vocab)  # fallback: recreate layer/model","preventionTips":["Never mutate vocabulary after first call; rebuild instead","Pin output width with pad_to_max_tokens=True when vocab may grow"],"tags":["keras","vocabulary","frozen-state","shape-validation"],"backgroundTag":"shape-frozen-after-first-use","analyzedSha":"7a34a03db60bf60042242d6a556fc3be119046a5","analyzedAt":"2026-08-25T21:25:25.994Z","schemaVersion":2},"datasetVersion":"2026-08-26T02:17:13.382Z"}