keras-team/keras · error · ValueError

Specify a dimension (`num_words` argument), or fit on some t

Error message

Specify a dimension (`num_words` argument), or fit on some text data first.

What it means

Tokenizer.sequences_to_matrix (and texts_to_matrix) must know the output vector width. If num_words was never set and the tokenizer was never fitted (empty word_index), there is no vocabulary size to infer, so it raises this ValueError.

Source

Thrown at keras/src/legacy/preprocessing/text.py:243

                        if oov_token_index is not None:
                            vect.append(self.index_word[oov_token_index])
                    else:
                        vect.append(word)
                elif self.oov_token is not None:
                    vect.append(self.index_word[oov_token_index])
            vect = " ".join(vect)
            yield vect

    def texts_to_matrix(self, texts, mode="binary"):
        sequences = self.texts_to_sequences(texts)
        return self.sequences_to_matrix(sequences, mode=mode)

    def sequences_to_matrix(self, sequences, mode="binary"):
        if not self.num_words:
            if self.word_index:
                num_words = len(self.word_index) + 1
            else:
                raise ValueError(
                    "Specify a dimension (`num_words` argument), "
                    "or fit on some text data first."
                )
        else:
            num_words = self.num_words

        if mode == "tfidf" and not self.document_count:
            raise ValueError(
                "Fit the Tokenizer on some data before using tfidf mode."
            )

        x = np.zeros((len(sequences), num_words))
        for i, seq in enumerate(sequences):
            if not seq:
                continue
            counts = collections.defaultdict(int)
            for j in seq:
                if j >= num_words:

View on GitHub (pinned to 7a34a03db6)

Solutions

1. Call fit_on_texts(training_texts) before texts_to_matrix/sequences_to_matrix
2. Or set the dimension explicitly: Tokenizer(num_words=10000)
3. Persist the fitted tokenizer (pickle or to_json) and reload it instead of recreating

Example fix

# before
tok = Tokenizer()
tok.texts_to_matrix(['hello world'])
# after
tok = Tokenizer()
tok.fit_on_texts(['hello world', 'more text'])
tok.texts_to_matrix(['hello world'])

if not tok.num_words and not tok.word_index:
    tok.fit_on_texts(corpus)
# or: assert tok.num_words or tok.word_index

def tokenizer_ready(tok):
    return bool(tok.num_words or tok.word_index)

When it happens

Trigger: Creating a fresh Tokenizer() with no num_words and calling texts_to_matrix(texts) before fit_on_texts; reusing an unpickled tokenizer whose fit state was lost.

Common situations: Notebook workflows where fit_on_texts is skipped or run on a different Tokenizer instance; services that build the tokenizer per request.

Related errors

• Unrecognized keyword arguments: {str(kwargs)}
• Fit the Tokenizer on some data before using tfidf mode.
• Unknown vectorization mode:
• Unknown activation function '{activation}' cannot be seriali
• Could not interpret activation function identifier: {identif

AI-assisted analysis of keras-team/keras@7a34a03db6 (2026-08-25). Data as JSON: /api/errors/45ec1c2a418c075e. Report an issue: GitHub.