keras-team/keras · error · ValueError

`idf_weights` must be set if output_mode is 'tf_idf'.

Error message

`idf_weights` must be set if output_mode is 'tf_idf'.

What it means

set_vocabulary() on a TF-IDF layer must receive idf_weights alongside the tokens, because the lookup table and the weight vector are installed together.

Source

Thrown at keras/src/layers/preprocessing/index_lookup.py:434

        instead of analyzing a dataset through `adapt`. It should be used
        whenever the vocab (and optionally document frequency) information is
        already known.  If vocabulary data is already present in the layer, this
        method will replace it.

        Args:
            vocabulary: Either an array or a string path to a text file.
                If passing an array, can pass a tuple, list,
                1D numpy array, or 1D tensor containing the vocbulary terms.
                If passing a file path, the file should contain one line
                per term in the vocabulary.
            idf_weights: A tuple, list, 1D numpy array, or 1D tensor
                of inverse document frequency weights with equal
                length to vocabulary. Must be set if `output_mode`
                is `"tf_idf"`. Should not be set otherwise.
        """
        if self.output_mode == "tf_idf":
            if idf_weights is None:
                raise ValueError(
                    "`idf_weights` must be set if output_mode is 'tf_idf'."
                )
        elif idf_weights is not None:
            raise ValueError(
                "`idf_weights` should only be set if output_mode is "
                f"`'tf_idf'`. Received: output_mode={self.output_mode} "
                f"and idf_weights={idf_weights}"
            )

        if isinstance(vocabulary, str):
            if serialization_lib.in_safe_mode():
                raise ValueError(
                    "Requested the loading of a vocabulary file outside of the "
                    "model archive. This carries a potential risk of loading "
                    "arbitrary and sensitive files and thus it is disallowed "
                    "by default. If you trust the source of the artifact, you "
                    "can override this error by passing `safe_mode=False` to "
                    "the loading function, or calling "

View on GitHub (pinned to 7a34a03db6)

Solutions

  1. Compute idf weights (e.g. fit sklearn TfidfVectorizer on the same corpus) and pass them: `set_vocabulary(vocab, idf_weights=w)`.
  2. If it happens on load, re-save the model with the current Keras so idf_weights serialize with the layer.

Example fix

# before
tfidf_layer.set_vocabulary(vocab)

# after
tfidf_layer.set_vocabulary(vocab, idf_weights=idf)
Defensive patterns

Strategy: validation

Validate before calling

if layer.output_mode == 'tf_idf':
    assert idf is not None, 'tf_idf layer needs idf_weights'
    layer.set_vocabulary(vocab, idf_weights=idf)
else:
    layer.set_vocabulary(vocab)

Try / catch

try:
    layer.set_vocabulary(vocab, idf_weights=idf)
except ValueError as e:
    raise RuntimeError('vocabulary setup failed: %s' % e) from e

Prevention

When it happens

Trigger: `layer.set_vocabulary(vocab)` on a layer built with output_mode='tf_idf'. Also raised from load_assets when deserializing a model whose idf weights were not saved.

Common situations: Calling adapt()/set_vocabulary with a token-only dataset (adapt cannot compute idf); loading models saved by an older Keras that dropped idf weights.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of keras-team/keras@7a34a03db6 (2026-08-25). Data as JSON: /api/errors/139fe53fc3e4dd2b. Report an issue: GitHub.