{"record":{"id":"a3c6357a6b4a8ae8","repo":"keras-team/keras","slug":"when-specifying-the-vocabulary-argument-in-tf-i","errorCode":null,"errorMessage":"When specifying the `vocabulary` argument, in TF-IDF output mode, the `idf_weights` argument must also be provided.","messagePattern":"When specifying the `vocabulary` argument, in TF-IDF output mode, the `idf_weights` argument must also be provided\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"keras/src/layers/preprocessing/index_lookup.py","lineNumber":292,"sourceCode":"            elif self.num_oov_indices == 1:\n                # If there is only one OOV index, we can set that index as the\n                # default value of the index_lookup table.\n                self._default_value = self._oov_start_index()\n            else:\n                # If we have multiple OOV values, we need to do a further\n                # hashing step; to make this easier, we set the OOV value to -1.\n                # (This lets us do a vectorized add and cast to boolean to\n                # determine locations where we need to do extra hashing.)\n                self._default_value = -1\n        if self.mask_token is not None:\n            self._mask_key = tf.convert_to_tensor(mask_key, self._key_dtype)\n            self._mask_value = tf.convert_to_tensor(\n                mask_value, self._value_dtype\n            )\n\n        if self.output_mode == \"tf_idf\":\n            if self._has_input_vocabulary and idf_weights is None:\n                raise ValueError(\n                    \"When specifying the `vocabulary` argument, \"\n                    \"in TF-IDF output mode, the `idf_weights` argument \"\n                    \"must also be provided.\"\n                )\n            if idf_weights is not None:\n                self.idf_weights = tf.Variable(\n                    idf_weights,\n                    dtype=backend.floatx(),\n                    trainable=False,\n                )\n                self.idf_weights_const = self.idf_weights.value()\n\n        if vocabulary is not None:\n            self.set_vocabulary(vocabulary, idf_weights)\n        else:\n            # When restoring from a keras SavedModel, the loading code will\n            # expect to find and restore a lookup_table attribute on the layer.\n            # This table needs to be uninitialized as a StaticHashTable cannot","sourceCodeStart":274,"sourceCodeEnd":310,"githubUrl":"https://github.com/keras-team/keras/blob/7a34a03db60bf60042242d6a556fc3be119046a5/keras/src/layers/preprocessing/index_lookup.py#L274-L310","documentation":"When a vocabulary is passed directly in the constructor with output_mode='tf_idf', the layer builds its lookup table immediately, so the idf weights must be supplied at the same moment. A frozen TF-IDF vocabulary has no later adapt path.","triggerScenarios":"`IndexLookup(vocabulary=vocab, output_mode='tf_idf')` with no `idf_weights`.","commonSituations":"Porting a fitted sklearn TfidfVectorizer: importing `vocabulary_` but forgetting `idf_`; switching a working int-mode layer to tf_idf while keeping the same constructor call.","solutions":["Pass `idf_weights` together with `vocabulary` in the constructor.","Or leave `vocabulary=None` and call `set_vocabulary(vocab, idf_weights=w)` afterwards.","When porting sklearn, use `vectorizer.idf_` for the weights."],"exampleFix":"# before\nlayer = IndexLookup(vocabulary=vocab, output_mode='tf_idf')\n\n# after\nlayer = IndexLookup(vocabulary=vocab, idf_weights=idf, output_mode='tf_idf')","handlingStrategy":"validation","validationCode":"if output_mode == 'tf_idf' and vocabulary is not None:\n    assert idf_weights is not None, 'tf_idf + vocabulary requires idf_weights'\nlayer = IndexLookup(vocabulary=vocabulary, idf_weights=idf_weights, output_mode=output_mode)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["When porting sklearn, always import vocabulary_ and idf_ as a pair.","Prefer set_vocabulary(vocab, idf_weights=...) over constructor vocabulary for tf_idf layers."],"tags":["keras","preprocessing","index-lookup","tf-idf","missing-argument"],"backgroundTag":"missing-required-argument","analyzedSha":"7a34a03db60bf60042242d6a556fc3be119046a5","analyzedAt":"2026-08-25T21:25:25.994Z","schemaVersion":2},"datasetVersion":"2026-08-26T02:17:13.382Z"}