What causes "Unknown word vector not specified in the word vector file" in stanfordnlp/CoreNLP?

Add a line for the unknown token (e.g. "*UNKNOWN*" followed by numHid values) to the word vector file

stanfordnlp/CoreNLP · error · RuntimeException

Unknown word vector not specified in the word vector file

Error message

Unknown word vector not specified in the word vector file

What it means

readWordVectors loads pretrained vectors from op.wordVectorFile and then looks up op.unkWord to serve as the vector for out-of-vocabulary words (stored under UNKNOWN_WORD). If the vector file does not contain an entry for the unkWord token, the unknown word vector is null and it throws RuntimeException.

Solutions

Add a line for the unknown token (e.g. "*UNKNOWN*" followed by numHid values) to the word vector file
Set op.unkWord to a token that actually exists in your word vector file
Generate and append a random or zero vector entry for the unknown word before training

Example fix

// before
RNNOptions op = new RNNOptions();
op.wordVectorFile = "glove.txt"; // has no *UNKNOWN* entry
SentimentModel model = new SentimentModel(op, trees); // throws
// after
echo "*UNKNOWN* 0.01 0.02 ... (25 values)" >> glove.txt
RNNOptions op = new RNNOptions();
op.wordVectorFile = "glove.txt";
op.unkWord = "*UNKNOWN*";
SentimentModel model = new SentimentModel(op, trees);

Defensive patterns

Strategy: validation

Validate before calling

// Verify the unknown token exists in the embedding file before model construction
String unk = op.unkWord; // default "*UNKNOWN*"
boolean found = false;
try (BufferedReader r = Files.newBufferedReader(Paths.get(op.wordVectorFile))) {
  String line;
  while ((line = r.readLine()) != null) {
    if (line.startsWith(unk + " ")) { found = true; break; }
  }
}
if (!found) throw new IllegalStateException("Word vector file missing unknown token: " + unk);

Try / catch

try {
  model = new SentimentModel(op, trees);
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("Unknown word vector")) {
    op.unkWord = resolveExistingUnkToken(op.wordVectorFile); // pick a token present in the file
    model = new SentimentModel(op, trees);
  } else { throw e; }
}

Prevention

Include an explicit *UNKNOWN* vector in every embedding file you ship
Keep op.unkWord in sync with the token actually present in the vector file
Script your embedding preprocessing to always append an unknown-word entry
Smoke-test SentimentModel construction early in CI with the real vector file

When it happens

Trigger: Constructing a SentimentModel with a wordVectorFile that lacks a line for the token configured in op.unkWord (default "*UNKNOWN*").

Common situations: Using custom word2vec/GloVe embeddings that don't include the default unknown token; changing op.unkWord without adding that token to the embeddings; trimming rare tokens from the embedding file during preprocessing.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/20b9c6d956fcab89. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/sentiment/SentimentModel.java:517

      wordVectors.put(word, vector);
    }
  }

  void readWordVectors() {
    Embedding embedding = new Embedding(op.wordVectors, op.numHid);
    this.wordVectors = Generics.newTreeMap();
//    Map<String, SimpleMatrix> rawWordVectors = NeuralUtils.readRawWordVectors(op.wordVectors, op.numHid);
//    for (String word : rawWordVectors.keySet()) {
    for (String word : embedding.keySet()) {
      // TODO: factor out unknown word vector code from DVParser
      wordVectors.put(word, embedding.get(word));
    }

    String unkWord = op.unkWord;
    SimpleMatrix unknownWordVector = wordVectors.get(unkWord);
    wordVectors.put(UNKNOWN_WORD, unknownWordVector);
    if (unknownWordVector == null) {
      throw new RuntimeException("Unknown word vector not specified in the word vector file");
    }

  }

  public int totalParamSize() {
    int totalSize = 0;
    // binaryTensorSize was set to 0 if useTensors=false
    totalSize = numBinaryMatrices * (binaryTransformSize + binaryClassificationSize + binaryTensorSize);
    totalSize += numUnaryMatrices * unaryClassificationSize;
    totalSize += wordVectors.size() * numHid;
    return totalSize;
  }

  public double[] paramsToVector() {
    int totalSize = totalParamSize();
    return NeuralUtils.paramsToVector(totalSize, binaryTransform.valueIterator(), binaryClassification.valueIterator(), SimpleTensor.iteratorSimpleMatrix(binaryTensors.valueIterator()), unaryClassification.values().iterator(), wordVectors.values().iterator());
  }

View on GitHub (pinned to 1b7edd19c4)