keras-team/keras · error · ValueError

Fit the Tokenizer on some data before using tfidf mode.

Error message

Fit the Tokenizer on some data before using tfidf mode.

What it means

tfidf mode needs corpus-level document frequencies (index_docs and document_count) computed during fit_on_texts. Requesting mode='tfidf' while document_count is 0 raises this ValueError, even if num_words is set.

Source

Thrown at keras/src/legacy/preprocessing/text.py:251

    def texts_to_matrix(self, texts, mode="binary"):
        sequences = self.texts_to_sequences(texts)
        return self.sequences_to_matrix(sequences, mode=mode)

    def sequences_to_matrix(self, sequences, mode="binary"):
        if not self.num_words:
            if self.word_index:
                num_words = len(self.word_index) + 1
            else:
                raise ValueError(
                    "Specify a dimension (`num_words` argument), "
                    "or fit on some text data first."
                )
        else:
            num_words = self.num_words

        if mode == "tfidf" and not self.document_count:
            raise ValueError(
                "Fit the Tokenizer on some data before using tfidf mode."
            )

        x = np.zeros((len(sequences), num_words))
        for i, seq in enumerate(sequences):
            if not seq:
                continue
            counts = collections.defaultdict(int)
            for j in seq:
                if j >= num_words:
                    continue
                counts[j] += 1
            for j, c in list(counts.items()):
                if mode == "count":
                    x[i][j] = c
                elif mode == "freq":
                    x[i][j] = c / len(seq)
                elif mode == "binary":

View on GitHub (pinned to 7a34a03db6)

Solutions

1. Call fit_on_texts on the training corpus before using mode='tfidf'
2. Save and reload the fitted tokenizer rather than rebuilding it
3. If document statistics are unavailable, use mode='count' or 'binary'

Example fix

# before
tok = Tokenizer(num_words=5000)
X = tok.texts_to_matrix(texts, mode='tfidf')
# after
tok = Tokenizer(num_words=5000)
tok.fit_on_texts(train_texts)
X = tok.texts_to_matrix(texts, mode='tfidf')

if mode == 'tfidf' and not tok.document_count:
    tok.fit_on_texts(corpus)

def can_tfidf(tok):
    return tok.document_count > 0

When it happens

Trigger: Tokenizer(num_words=5000).texts_to_matrix(texts, mode='tfidf') on a never-fitted tokenizer.

Common situations: Assuming num_words alone suffices; inference service reconstructs a fresh tokenizer and calls tfidf vectorization without loading fit state.

Related errors

• Unrecognized keyword arguments: {str(kwargs)}
• Specify a dimension (`num_words` argument), or fit on some t
• Unknown vectorization mode:
• Unknown activation function '{activation}' cannot be seriali
• Could not interpret activation function identifier: {identif

AI-assisted analysis of keras-team/keras@7a34a03db6 (2026-08-25). Data as JSON: /api/errors/2460ef1789c1e9aa. Report an issue: GitHub.