{"record":{"id":"819085e2a31af07b","repo":"stanfordnlp/CoreNLP","slug":"unknown-word-vector-not-specified-in-the-word-vect","errorCode":null,"errorMessage":"Unknown word vector not specified in the word vector file","messagePattern":"Unknown word vector not specified in the word vector file","errorType":"exception","errorClass":"java.lang.RuntimeException","httpStatus":null,"severity":"critical","filePath":"src/edu/stanford/nlp/parser/dvparser/DVModel.java","lineNumber":537,"sourceCode":"\n      if (op.trainOptions.unknownChinesePercentVector && CHINESE_PERCENT_PATTERN.matcher(word).matches()) {\n        ++chinesePercentCount;\n        if (unknownChinesePercentVector == null) {\n          unknownChinesePercentVector = new SimpleMatrix(vector);\n        } else {\n          unknownChinesePercentVector = unknownChinesePercentVector.plus(vector);\n        }\n      }\n    }\n\n    String unkWord = op.trainOptions.unkWord;\n    if (op.wordFunction != null) {\n      unkWord = op.wordFunction.apply(unkWord);\n    }\n    SimpleMatrix unknownWordVector = wordVectors.get(unkWord);\n    wordVectors.put(UNKNOWN_WORD, unknownWordVector);\n    if (unknownWordVector == null) {\n      throw new RuntimeException(\"Unknown word vector not specified in the word vector file\");\n    }\n\n    if (op.trainOptions.unknownNumberVector) {\n      if (numberCount > 0) {\n        unknownNumberVector = unknownNumberVector.divide(numberCount);\n      } else {\n        unknownNumberVector = new SimpleMatrix(unknownWordVector);\n      }\n      wordVectors.put(UNKNOWN_NUMBER, unknownNumberVector);\n    }\n\n    if (op.trainOptions.unknownCapsVector) {\n      if (capsCount > 0) {\n        unknownCapsVector = unknownCapsVector.divide(capsCount);\n      } else {\n        unknownCapsVector = new SimpleMatrix(unknownWordVector);\n      }\n      wordVectors.put(UNKNOWN_CAPS, unknownCapsVector);","sourceCodeStart":519,"sourceCodeEnd":555,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/parser/dvparser/DVModel.java#L519-L555","documentation":"DVModel.readWordVectors looks up the unknown-word token (after applying op.wordFunction if set) in the loaded word vector map and stores it under UNKNOWN_WORD. If the unknown word has no vector in the supplied word vector file it throws RuntimeException, because the model cannot represent words missing from the vocabulary at parse time.","triggerScenarios":"Loading a DVModel with a word vector file that does not contain an entry for the configured unknown word token (commonly 'UNK' after any wordFunction transformation), or using a wordFunction that maps the unk token to a word absent from the vectors.","commonSituations":"Swapping in a pretrained embedding file (word2vec/glove) that lacks the UNK token; applying a lowercase wordFunction when the vector file's UNK entry is uppercase ('UNK' vs 'unk'); truncated or partial vector files.","solutions":["Add an entry for the unknown word token (e.g. UNK) to the word vector file","Verify what unkWord resolves to after op.wordFunction and ensure exactly that key exists in the vectors","Check for case/whitespace mismatches between the file's keys and the unk token"],"exampleFix":"// before\nglove.txt contains: the 0.1 ... (no UNK line)\n// after\nprintf 'UNK 0.0 0.0 0.0 ...\\n' >> vectors.txt  # or set op.wordFunction so unkWord matches an existing key","handlingStrategy":"try-catch","validationCode":"Map<String, double[]> vectors = loadWordVectors(dvWordVectorsFile);\nString unk = op.wordFunction != null ? op.wordFunction.apply(\"UNK\") : \"UNK\";\nif (!vectors.containsKey(unk)) {\n    throw new IllegalArgumentException(\"Vector file missing unknown-word entry: \" + unk);\n}","typeGuard":null,"tryCatchPattern":"try {\n    DVModel model = new DVModel(op, stateIndex, wordlist, dvWordVectorsFile);\n} catch (RuntimeException e) {\n    if (e.getMessage().contains(\"Unknown word vector\")) {\n        System.err.println(\"Add an UNK entry (matching op.wordFunction output) to the vector file\");\n    }\n}","preventionTips":["Ensure the word vector file contains the unknown token (typically UNK)","Verify case after applying op.wordFunction (e.g. lowercase vs UNK)","Sanity-check pretrained embedding files for required special tokens before use"],"tags":["word-vectors","model-loading","stanford-corenlp"],"backgroundTag":"resource-not-found","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}