{"record":{"id":"819f329372c95c1e","repo":"stanfordnlp/CoreNLP","slug":"word-vectors-file-has-dimension-too-small-for-requ","errorCode":null,"errorMessage":"Word vectors file has dimension too small for requested numHid of ","messagePattern":"Word vectors file has dimension too small for requested numHid of ","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/neural/Embedding.java","lineNumber":157,"sourceCode":"      if(word.equals(\"</s>\")){\n        word = END_WORD;\n      }\n\n      int dimOfWords = lineSplit.length - 1;\n      if (embeddingSize <= 0) {\n        embeddingSize = dimOfWords;\n        log.info(\"  detected embedding size = \" + dimOfWords);\n      }\n      // the first entry is the word itself\n      // the other entries will all be entries in the word vector\n      if (dimOfWords > embeddingSize) {\n        if (!warned) {\n          warned = true;\n          log.info(\"WARNING: Dimensionality of numHid parameter and word vectors do not match, deleting word vector dimensions to fit!\");\n        }\n        dimOfWords = embeddingSize;\n      } else if (dimOfWords < embeddingSize) {\n        throw new RuntimeException(\"Word vectors file has dimension too small for requested numHid of \" + embeddingSize);\n      }\n      double[][] vec = new double[dimOfWords][1];\n      for (int i = 1; i <= dimOfWords; i++) {\n        vec[i-1][0] = Double.parseDouble(lineSplit[i]);\n      }\n      SimpleMatrix vector = new SimpleMatrix(vec);\n      wordVectors.put(word, vector);\n\n      numWords++;\n    }\n    log.info(\"  num words = \" + numWords);\n  }\n\n  /**\n   * This method takes as input two files: wordFile (one word per line) and a raw word vector file\n   * with a given expected size, and returns a map of word to vector.\n   * <p>\n   * The word vector file should be in the format <br>","sourceCodeStart":139,"sourceCodeEnd":175,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/neural/Embedding.java#L139-L175","documentation":"Embedding.loadWordVectors parses a word-vector text file and compares each row's numeric width (dimOfWords) to the requested embedding size (embeddingSize / numHid). If the file's vectors are narrower than the requested dimension, it cannot pad them, so it throws this RuntimeException. It is thrown only when the file dimension is strictly smaller; larger dimensions are truncated with a warning instead.","triggerScenarios":"Calling an Embedding constructor (or loadWordVectors) with a numHid/embeddingSize larger than the number of numeric columns in the supplied word-vector file, e.g. word2vec vectors of dim 50 with numHid=100.","commonSituations":"Mismatch between a models/config file that says numHid=100 and a pre-trained vectors file of dimension 50 or 300; downloading vectors of the wrong dimension for the task.","solutions":["Regenerate/download a word-vector file whose dimension equals (or exceeds) the requested numHid","Lower the numHid/embeddingSize parameter to match the vector file's dimension","Check the vectors file header (first line is often 'vocab dim') and set numHid accordingly"],"exampleFix":"// before\nEmbedding embedding = new Embedding(\"vectors-50d.txt\", 100);\n// after\nEmbedding embedding = new Embedding(\"vectors-50d.txt\", 50); // match file dimension","handlingStrategy":"validation","validationCode":"int fileDim = getVectorDimension(vectorsFile); // parse header or first line\nif (fileDim < numHid) throw new IllegalArgumentException(\"vectors dim \" + fileDim + \" < numHid \" + numHid);","typeGuard":null,"tryCatchPattern":"try { new Embedding(file, numHid); } catch (RuntimeException e) { if (e.getMessage().startsWith(\"Word vectors file has dimension too small\")) { /* fix numHid or file */ } else throw e; }","preventionTips":["Parse the vectors file header (vocab size, dimension) and set numHid from it","Keep numHid in the same config key as the vectors file path","Add a startup assertion comparing file dimension to numHid"],"tags":["java","stanford-nlp","embeddings","dimension-mismatch"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}