{"record":{"id":"45ec1c2a418c075e","repo":"keras-team/keras","slug":"specify-a-dimension-num-words-argument-or-fit","errorCode":null,"errorMessage":"Specify a dimension (`num_words` argument), or fit on some text data first.","messagePattern":"Specify a dimension \\(`num_words` argument\\), or fit on some text data first\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"keras/src/legacy/preprocessing/text.py","lineNumber":243,"sourceCode":"                        if oov_token_index is not None:\n                            vect.append(self.index_word[oov_token_index])\n                    else:\n                        vect.append(word)\n                elif self.oov_token is not None:\n                    vect.append(self.index_word[oov_token_index])\n            vect = \" \".join(vect)\n            yield vect\n\n    def texts_to_matrix(self, texts, mode=\"binary\"):\n        sequences = self.texts_to_sequences(texts)\n        return self.sequences_to_matrix(sequences, mode=mode)\n\n    def sequences_to_matrix(self, sequences, mode=\"binary\"):\n        if not self.num_words:\n            if self.word_index:\n                num_words = len(self.word_index) + 1\n            else:\n                raise ValueError(\n                    \"Specify a dimension (`num_words` argument), \"\n                    \"or fit on some text data first.\"\n                )\n        else:\n            num_words = self.num_words\n\n        if mode == \"tfidf\" and not self.document_count:\n            raise ValueError(\n                \"Fit the Tokenizer on some data before using tfidf mode.\"\n            )\n\n        x = np.zeros((len(sequences), num_words))\n        for i, seq in enumerate(sequences):\n            if not seq:\n                continue\n            counts = collections.defaultdict(int)\n            for j in seq:\n                if j >= num_words:","sourceCodeStart":225,"sourceCodeEnd":261,"githubUrl":"https://github.com/keras-team/keras/blob/7a34a03db60bf60042242d6a556fc3be119046a5/keras/src/legacy/preprocessing/text.py#L225-L261","documentation":"Tokenizer.sequences_to_matrix (and texts_to_matrix) must know the output vector width. If num_words was never set and the tokenizer was never fitted (empty word_index), there is no vocabulary size to infer, so it raises this ValueError.","triggerScenarios":"Creating a fresh Tokenizer() with no num_words and calling texts_to_matrix(texts) before fit_on_texts; reusing an unpickled tokenizer whose fit state was lost.","commonSituations":"Notebook workflows where fit_on_texts is skipped or run on a different Tokenizer instance; services that build the tokenizer per request.","solutions":["Call fit_on_texts(training_texts) before texts_to_matrix/sequences_to_matrix","Or set the dimension explicitly: Tokenizer(num_words=10000)","Persist the fitted tokenizer (pickle or to_json) and reload it instead of recreating"],"exampleFix":"# before\ntok = Tokenizer()\ntok.texts_to_matrix(['hello world'])\n# after\ntok = Tokenizer()\ntok.fit_on_texts(['hello world', 'more text'])\ntok.texts_to_matrix(['hello world'])","handlingStrategy":"validation","validationCode":"if not tok.num_words and not tok.word_index:\n    tok.fit_on_texts(corpus)\n# or: assert tok.num_words or tok.word_index","typeGuard":"def tokenizer_ready(tok):\n    return bool(tok.num_words or tok.word_index)","tryCatchPattern":null,"preventionTips":["Fit immediately after constructing; persist fitted tokenizers and reload them"],"tags":["keras","text","tokenizer","fit-before-transform"],"backgroundTag":"unfitted-transformer","analyzedSha":"7a34a03db60bf60042242d6a556fc3be119046a5","analyzedAt":"2026-08-25T21:25:25.994Z","schemaVersion":2},"datasetVersion":"2026-08-26T02:17:13.382Z"}