stanfordnlp/CoreNLP · error · IOException

Could not read string buffer fully!

Error message

Could not read string buffer fully!

What it means

VectorMap.deserialize reads a serialized map of string keys to float vectors: it reads a key's length (keyIntType), then attempts to read that many bytes in one call. If read() returns fewer bytes than requested, the stream is truncated/corrupt, so it throws this IOException. Note InputStream.read may also legitimately return short reads, so this also fires on partial network/stream reads.

Solutions

1. Regenerate/rewrite the serialized VectorMap file with the same library version and keyIntType
2. Wrap the stream in DataInputStream / use readFully() instead of a single read() call to tolerate short reads
3. Verify the source stream/file is complete (checksum, file size) before deserializing

Example fix

// before
if (dataIn.read(buffer, 0, strlen) != strlen) { throw new IOException("Could not read string buffer fully!"); }
// after
dataIn.readFully(buffer);

if (file.length() < expectedMinBytes) throw new IOException("serialized VectorMap truncated");

try { return VectorMap.deserialize(in, dim, size, keyIntType); } catch (IOException e) { if (e.getMessage().contains("Could not read string buffer fully")) { throw new CorruptModelException("re-download/regenerate serialized vectors", e); } throw e; }

When it happens

Trigger: Deserializing a VectorMap from a truncated or corrupt byte stream/file; reading from a socket or stream where read() returns fewer bytes than the declared key length.

Common situations: Version mismatch between serializer and deserializer (different keyIntType or dim); truncated file transfer; reading a file written with a different VectorMap serialization format.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

• edu.stanford.nlp.io.RuntimeIOException
• Could not open temporary feature index file for reading.
• Serializing classifier to
• Failed to load segmenter
• don't know how to get Reader from class

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/22d60082fdc50fe7. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/neural/VectorMap.java:191

   */
  public static VectorMap deserialize(InputStream in) throws IOException {
    DataInputStream dataIn = new DataInputStream(in);

    // Read the max key length
    itype keyIntType = itype.getType(dataIn.readInt());
    // Read the vector dimensionality
    int dim = dataIn.readInt();
    // Read the size of the dataset
    int size = dataIn.readInt();

    // Read the vectors
    VectorMap vectors = new VectorMap();
    for (int i = 0; i < size; ++i) {
      // Read the key
      int strlen = keyIntType.read(dataIn);
      byte[] buffer = new byte[strlen];
      if (dataIn.read(buffer, 0, strlen) != strlen) {
        throw new IOException("Could not read string buffer fully!");
      }
      String key = new String(buffer);
      // Read the vector
      float[] vector = new float[dim];
      for (int k = 0; k < vector.length; ++k) {
        vector[k] = toFloat(dataIn.readShort());
      }
      // Add the key/value
      vectors.put(key, vector);
    }
    return vectors;
  }


  /**
   * Read the Word2Vec word vector flat txt file.
   *
   * @param file The word2vec text file.

View on GitHub (pinned to 1b7edd19c4)