stanfordnlp/CoreNLP · critical · java.lang.RuntimeException

Unknown word vector not specified in the word vector file

Error message

Unknown word vector not specified in the word vector file

What it means

DVModel.readWordVectors looks up the unknown-word token (after applying op.wordFunction if set) in the loaded word vector map and stores it under UNKNOWN_WORD. If the unknown word has no vector in the supplied word vector file it throws RuntimeException, because the model cannot represent words missing from the vocabulary at parse time.

Solutions

1. Add an entry for the unknown word token (e.g. UNK) to the word vector file
2. Verify what unkWord resolves to after op.wordFunction and ensure exactly that key exists in the vectors
3. Check for case/whitespace mismatches between the file's keys and the unk token

Example fix

// before
glove.txt contains: the 0.1 ... (no UNK line)
// after
printf 'UNK 0.0 0.0 0.0 ...\n' >> vectors.txt  # or set op.wordFunction so unkWord matches an existing key

Map<String, double[]> vectors = loadWordVectors(dvWordVectorsFile);
String unk = op.wordFunction != null ? op.wordFunction.apply("UNK") : "UNK";
if (!vectors.containsKey(unk)) {
    throw new IllegalArgumentException("Vector file missing unknown-word entry: " + unk);
}

try {
    DVModel model = new DVModel(op, stateIndex, wordlist, dvWordVectorsFile);
} catch (RuntimeException e) {
    if (e.getMessage().contains("Unknown word vector")) {
        System.err.println("Add an UNK entry (matching op.wordFunction output) to the vector file");
    }
}

When it happens

Trigger: Loading a DVModel with a word vector file that does not contain an entry for the configured unknown word token (commonly 'UNK' after any wordFunction transformation), or using a wordFunction that maps the unk token to a word absent from the vectors.

Common situations: Swapping in a pretrained embedding file (word2vec/glove) that lacks the UNK token; applying a lowercase wordFunction when the vector file's UNK entry is uppercase ('UNK' vs 'unk'); truncated or partial vector files.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• format error in embeddings
• format error unexpected featureFactory line:
• Expected parsers with DVModel embedded
• This parser does not contain a DVModel reranker
• Error loading classifier from

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/819085e2a31af07b. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/parser/dvparser/DVModel.java:537

      if (op.trainOptions.unknownChinesePercentVector && CHINESE_PERCENT_PATTERN.matcher(word).matches()) {
        ++chinesePercentCount;
        if (unknownChinesePercentVector == null) {
          unknownChinesePercentVector = new SimpleMatrix(vector);
        } else {
          unknownChinesePercentVector = unknownChinesePercentVector.plus(vector);
        }
      }
    }

    String unkWord = op.trainOptions.unkWord;
    if (op.wordFunction != null) {
      unkWord = op.wordFunction.apply(unkWord);
    }
    SimpleMatrix unknownWordVector = wordVectors.get(unkWord);
    wordVectors.put(UNKNOWN_WORD, unknownWordVector);
    if (unknownWordVector == null) {
      throw new RuntimeException("Unknown word vector not specified in the word vector file");
    }

    if (op.trainOptions.unknownNumberVector) {
      if (numberCount > 0) {
        unknownNumberVector = unknownNumberVector.divide(numberCount);
      } else {
        unknownNumberVector = new SimpleMatrix(unknownWordVector);
      }
      wordVectors.put(UNKNOWN_NUMBER, unknownNumberVector);
    }

    if (op.trainOptions.unknownCapsVector) {
      if (capsCount > 0) {
        unknownCapsVector = unknownCapsVector.divide(capsCount);
      } else {
        unknownCapsVector = new SimpleMatrix(unknownWordVector);
      }
      wordVectors.put(UNKNOWN_CAPS, unknownCapsVector);

View on GitHub (pinned to 1b7edd19c4)