keras-team/keras · error · RuntimeError

Cannot set a tensor vocabulary on layer

Error message

Cannot set a tensor vocabulary on layer {self.name} when not executing eagerly. Create this layer or call `set_vocabulary()` outside of any traced function.

What it means

Inside a traced function (tf.function graph, tf.data map, or Keras 3 multi-backend trace) there is no eager execution context, so set_vocabulary cannot convert tensor inputs — lookup table creation requires eager side effects. The layer raises RuntimeError to fail fast rather than bake a stale table into the graph.

Solutions

Move set_vocabulary or layer construction outside the traced function, calling it eagerly at setup time.
Convert tensors to numpy first and pass the array.
In tf.data pipelines, finish vocabulary setup before building the dataset, not inside a map function.

Example fix

# before
@tf.function
def setup(layer, vocab):
    layer.set_vocabulary(vocab)

# after
layer.set_vocabulary(vocab.numpy())  # eagerly, outside any trace

Defensive patterns

Strategy: validation

Validate before calling

import tensorflow as tf
if tf.is_tensor(vocab):
    assert tf.executing_eagerly(), 'set_vocabulary needs eager mode for tensors'
    vocab = vocab.numpy()
layer.set_vocabulary(vocab)

Type guard

def can_set_tensor_vocab() -> bool:
    import tensorflow as tf
    return tf.executing_eagerly()

Try / catch

try:
    layer.set_vocabulary(vocab)
except RuntimeError:
    layer.set_vocabulary(vocab.numpy())  # retry eagerly outside the trace

Prevention

Do all vocabulary setup eagerly at pipeline-build time, never inside tf.function or tf.data map.
Convert tensors to numpy before handing them to preprocessing layers.

When it happens

Trigger: Constructing `IndexLookup(vocabulary=tensor)` or calling `set_vocabulary(vocab_tensor)` inside a `@tf.function`, a tf.data `.map()` callback, or another traced function.

Common situations: Moving layer creation or adaptation into a compiled train step or an input pipeline; JAX and torch tracing paths in Keras 3.

Related errors

AI-assisted analysis of keras-team/keras@7a34a03db6 (2026-08-25). Data as JSON: /api/errors/e93143f5b06b8055. Report an issue: GitHub.

Appendix: source

Thrown at keras/src/layers/preprocessing/index_lookup.py:473

                )

            if not tf.io.gfile.exists(vocabulary):
                raise ValueError(
                    f"Vocabulary file {vocabulary} does not exist."
                )
            if self.output_mode == "tf_idf":
                raise ValueError(
                    "output_mode `'tf_idf'` does not support loading a "
                    "vocabulary from file."
                )
            self.lookup_table = self._lookup_table_from_file(vocabulary)
            self._record_vocabulary_size()
            return

        if not tf.executing_eagerly() and (
            tf.is_tensor(vocabulary) or tf.is_tensor(idf_weights)
        ):
            raise RuntimeError(
                f"Cannot set a tensor vocabulary on layer {self.name} "
                "when not executing eagerly. "
                "Create this layer or call `set_vocabulary()` "
                "outside of any traced function."
            )

        # TODO(mattdangerw): for better performance we should rewrite this
        # entire function to operate on tensors and convert vocabulary to a
        # tensor here.
        if tf.is_tensor(vocabulary):
            vocabulary = self._tensor_vocab_to_numpy(vocabulary)
        elif isinstance(vocabulary, (list, tuple)):
            vocabulary = np.array(vocabulary)
        if tf.is_tensor(idf_weights):
            idf_weights = idf_weights.numpy()
        elif isinstance(idf_weights, (list, tuple)):
            idf_weights = np.array(idf_weights)

View on GitHub (pinned to 7a34a03db6)