{"record":{"id":"22d60082fdc50fe7","repo":"stanfordnlp/CoreNLP","slug":"could-not-read-string-buffer-fully","errorCode":null,"errorMessage":"Could not read string buffer fully!","messagePattern":"Could not read string buffer fully!","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/neural/VectorMap.java","lineNumber":191,"sourceCode":"   */\n  public static VectorMap deserialize(InputStream in) throws IOException {\n    DataInputStream dataIn = new DataInputStream(in);\n\n    // Read the max key length\n    itype keyIntType = itype.getType(dataIn.readInt());\n    // Read the vector dimensionality\n    int dim = dataIn.readInt();\n    // Read the size of the dataset\n    int size = dataIn.readInt();\n\n    // Read the vectors\n    VectorMap vectors = new VectorMap();\n    for (int i = 0; i < size; ++i) {\n      // Read the key\n      int strlen = keyIntType.read(dataIn);\n      byte[] buffer = new byte[strlen];\n      if (dataIn.read(buffer, 0, strlen) != strlen) {\n        throw new IOException(\"Could not read string buffer fully!\");\n      }\n      String key = new String(buffer);\n      // Read the vector\n      float[] vector = new float[dim];\n      for (int k = 0; k < vector.length; ++k) {\n        vector[k] = toFloat(dataIn.readShort());\n      }\n      // Add the key/value\n      vectors.put(key, vector);\n    }\n    return vectors;\n  }\n\n\n  /**\n   * Read the Word2Vec word vector flat txt file.\n   *\n   * @param file The word2vec text file.","sourceCodeStart":173,"sourceCodeEnd":209,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/neural/VectorMap.java#L173-L209","documentation":"VectorMap.deserialize reads a serialized map of string keys to float vectors: it reads a key's length (keyIntType), then attempts to read that many bytes in one call. If read() returns fewer bytes than requested, the stream is truncated/corrupt, so it throws this IOException. Note InputStream.read may also legitimately return short reads, so this also fires on partial network/stream reads.","triggerScenarios":"Deserializing a VectorMap from a truncated or corrupt byte stream/file; reading from a socket or stream where read() returns fewer bytes than the declared key length.","commonSituations":"Version mismatch between serializer and deserializer (different keyIntType or dim); truncated file transfer; reading a file written with a different VectorMap serialization format.","solutions":["Regenerate/rewrite the serialized VectorMap file with the same library version and keyIntType","Wrap the stream in DataInputStream / use readFully() instead of a single read() call to tolerate short reads","Verify the source stream/file is complete (checksum, file size) before deserializing"],"exampleFix":"// before\nif (dataIn.read(buffer, 0, strlen) != strlen) { throw new IOException(\"Could not read string buffer fully!\"); }\n// after\ndataIn.readFully(buffer);","handlingStrategy":"try-catch","validationCode":"if (file.length() < expectedMinBytes) throw new IOException(\"serialized VectorMap truncated\");","typeGuard":null,"tryCatchPattern":"try { return VectorMap.deserialize(in, dim, size, keyIntType); } catch (IOException e) { if (e.getMessage().contains(\"Could not read string buffer fully\")) { throw new CorruptModelException(\"re-download/regenerate serialized vectors\", e); } throw e; }","preventionTips":["Use DataInputStream.readFully-style reads for exact-length fields","Verify checksums on serialized model files before deserializing","Serialize and deserialize with the same library version and keyIntType"],"tags":["java","stanford-nlp","serialization","io"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}