keras-team/keras · error · RuntimeError

Cannot set a tensor vocabulary on layer {self.name} when not

Error message

Cannot set a tensor vocabulary on layer {self.name} when not executing eagerly. Create this layer or call `set_vocabulary()` outside of any traced function.

What it means

Inside a traced function (tf.function graph, tf.data map, or Keras 3 multi-backend trace) there is no eager execution context, so set_vocabulary cannot convert tensor inputs — lookup table creation requires eager side effects. The layer raises RuntimeError to fail fast rather than bake a stale table into the graph.

Source

Thrown at keras/src/layers/preprocessing/index_lookup.py:473

                )

            if not tf.io.gfile.exists(vocabulary):
                raise ValueError(
                    f"Vocabulary file {vocabulary} does not exist."
                )
            if self.output_mode == "tf_idf":
                raise ValueError(
                    "output_mode `'tf_idf'` does not support loading a "
                    "vocabulary from file."
                )
            self.lookup_table = self._lookup_table_from_file(vocabulary)
            self._record_vocabulary_size()
            return

        if not tf.executing_eagerly() and (
            tf.is_tensor(vocabulary) or tf.is_tensor(idf_weights)
        ):
            raise RuntimeError(
                f"Cannot set a tensor vocabulary on layer {self.name} "
                "when not executing eagerly. "
                "Create this layer or call `set_vocabulary()` "
                "outside of any traced function."
            )

        # TODO(mattdangerw): for better performance we should rewrite this
        # entire function to operate on tensors and convert vocabulary to a
        # tensor here.
        if tf.is_tensor(vocabulary):
            vocabulary = self._tensor_vocab_to_numpy(vocabulary)
        elif isinstance(vocabulary, (list, tuple)):
            vocabulary = np.array(vocabulary)
        if tf.is_tensor(idf_weights):
            idf_weights = idf_weights.numpy()
        elif isinstance(idf_weights, (list, tuple)):
            idf_weights = np.array(idf_weights)

View on GitHub (pinned to 7a34a03db6)

Solutions

1. Move set_vocabulary or layer construction outside the traced function, calling it eagerly at setup time.
2. Convert tensors to numpy first and pass the array.
3. In tf.data pipelines, finish vocabulary setup before building the dataset, not inside a map function.

Example fix

# before
@tf.function
def setup(layer, vocab):
    layer.set_vocabulary(vocab)

# after
layer.set_vocabulary(vocab.numpy())  # eagerly, outside any trace

import tensorflow as tf
if tf.is_tensor(vocab):
    assert tf.executing_eagerly(), 'set_vocabulary needs eager mode for tensors'
    vocab = vocab.numpy()
layer.set_vocabulary(vocab)

def can_set_tensor_vocab() -> bool:
    import tensorflow as tf
    return tf.executing_eagerly()

try:
    layer.set_vocabulary(vocab)
except RuntimeError:
    layer.set_vocabulary(vocab.numpy())  # retry eagerly outside the trace

When it happens

Trigger: Constructing `IndexLookup(vocabulary=tensor)` or calling `set_vocabulary(vocab_tensor)` inside a `@tf.function`, a tf.data `.map()` callback, or another traced function.

Common situations: Moving layer creation or adaptation into a compiled train step or an input pipeline; JAX and torch tracing paths in Keras 3.

Related errors

• If set, `max_tokens` must be greater than 1. Received: max_t
• If pad_to_max_tokens is True, must set `max_tokens`. Receive
• `num_oov_indices` must be greater than or equal to 0. Receiv
• `salt` can only be used when `oov_method='farmhash'`. Receiv
• The `salt` argument for `IndexLookup` can only be a tuple of

AI-assisted analysis of keras-team/keras@7a34a03db6 (2026-08-25). Data as JSON: /api/errors/e93143f5b06b8055. Report an issue: GitHub.