keras-team/keras · error · RuntimeError

When using `output_mode={self.output_mode}` and `pad_to_max_

Error message

When using `output_mode={self.output_mode}` and `pad_to_max_tokens=False`, the vocabulary size cannot be changed after the layer is called. Old vocab size is {self._frozen_vocab_size}, new vocab size is {new_vocab_size}

What it means

After a layer with output_mode != 'int' and pad_to_max_tokens=False is first called, its vocabulary size is frozen because downstream output shapes depend on it. Any later operation that changes vocabulary_size() (set_vocabulary, re-adapt, loading new assets) triggers this RuntimeError from _ensure_vocab_size_unchanged.

Source

Thrown at keras/src/layers/preprocessing/index_lookup.py:1052

                f"When using `output_mode={self.output_mode}` "
                "and `pad_to_max_tokens=False`, "
                "you must set the layer's vocabulary before calling it. Either "
                "pass a `vocabulary` argument to the layer, or call `adapt` "
                "with some sample data."
            )

    def _ensure_vocab_size_unchanged(self):
        if self.output_mode == "int" or self.pad_to_max_tokens:
            return

        with tf.init_scope():
            new_vocab_size = self.vocabulary_size()

        if (
            self._frozen_vocab_size is not None
            and new_vocab_size != self._frozen_vocab_size
        ):
            raise RuntimeError(
                f"When using `output_mode={self.output_mode}` "
                "and `pad_to_max_tokens=False`, "
                "the vocabulary size cannot be changed after the layer is "
                f"called. Old vocab size is {self._frozen_vocab_size}, "
                f"new vocab size is {new_vocab_size}"
            )

    def _find_repeated_tokens(self, vocabulary):
        """Return all repeated tokens in a vocabulary."""
        vocabulary_set = set(vocabulary)
        if len(vocabulary) != len(vocabulary_set):
            return [
                item
                for item, count in collections.Counter(vocabulary).items()
                if count > 1
            ]
        else:
            return []

View on GitHub (pinned to 7a34a03db6)

Solutions

1. Recreate the layer (and rebuild/recompile the model) when the vocabulary must change
2. Freeze a fixed output width up front via max_tokens + pad_to_max_tokens=True
3. Keep the same vocabulary length; only update token->index content, not size

Example fix

// before
out = layer(x)
layer.set_vocabulary(bigger_vocab)  # RuntimeError
// after
layer = TextVectorization(output_mode='multi_hot', max_tokens=5000, pad_to_max_tokens=True)
layer.set_vocabulary(bigger_vocab)

if layer._frozen_vocab_size is not None and layer.vocabulary_size() != layer._frozen_vocab_size:
    layer = rebuild_layer_with_new_vocab()  # recreate instead of mutating

try:
    layer.set_vocabulary(new_vocab)
except RuntimeError:
    layer = build_fresh_layer(new_vocab)  # fallback: recreate layer/model

When it happens

Trigger: Calling the layer once, then calling set_vocabulary() with a different-length vocabulary, re-adapting, or deserializing new weights into a live layer; saving a model, then loading and continuing with an altered vocabulary.

Common situations: Incremental training that refreshes vocabularies between epochs; serving pipelines that hot-swap vocabulary assets on a warmed-up model.

Related errors

• The `weights` argument should be either `None` (random initi
• Expected mode to be one of `caffe`, `tf` or `torch`. Receive
• Vocabulary file {vocabulary} does not exist.
• Cannot set an empty vocabulary. Received: vocabulary={vocabu
• The passed vocabulary has at least one repeated term. Please

AI-assisted analysis of keras-team/keras@7a34a03db6 (2026-08-25). Data as JSON: /api/errors/6f0babc2ead6a2a9. Report an issue: GitHub.