apache/hadoop · error · InvalidObjectException

Stream data required

Error message

Stream data required

What it means

Thrown from RawPathHandle.readObjectNoData with InvalidObjectException when the Java serialization runtime initializes a RawPathHandle from a stream that contains the class descriptor but no object data (for example when readObject encounters the class in a stream where the instance fields were never written, or class-data reconciliation leaves no payload). RawPathHandle's fd is transient and must be rebuilt from stream bytes; an instance with no stream data is invalid, so construction is refused.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawPathHandle.java:116

      fd.slice().get(x);
      out.write(x);
    }
  }

  private void readObject(ObjectInputStream in)
      throws IOException, ClassNotFoundException {
    in.defaultReadObject();
    int len = in.readInt();
    if (len < 0 || len > MAX_SIZE) {
      throw new IOException("Illegal buffer length " + len);
    }
    byte[] x = new byte[len];
    in.readFully(x);
    fd = ByteBuffer.wrap(x);
  }

  private void readObjectNoData() throws ObjectStreamException {
    throw new InvalidObjectException("Stream data required");
  }

}

View on GitHub (pinned to 2add963021)

Solutions

1. Re-create the handle from the source FileSystem instead of deserializing the stale object
2. Pin matching Hadoop versions on both ends of the serialization boundary
3. Catch InvalidObjectException/ObjectStreamException and fall back to path-based access

try (ObjectInputStream in = new ObjectInputStream(src)) {
  handle = (PathHandle) in.readObject();
} catch (InvalidObjectException | ObjectStreamException e) {
  // "Stream data required" -> class/stream shape mismatch; refetch handle
  handle = fs.getPathHandle(fs.getFileStatus(p));
}

When it happens

Trigger: Deserializing an object graph where a RawPathHandle placeholder was written by a different class shape (stream class-data mismatch), or reflective tricks that trigger readObjectNoData (e.g., deserialize with a superclass/subclass swap).

Common situations: Version skew between the JVM that serialized the handle and the one deserializing it after RawPathHandle was recompiled with changed inheritance; corrupt or hand-crafted object streams.

Related errors

• Illegal buffer length " + len
• Can't read FileStatusProto with negative size of ${size}
• No URI in deserialized Path
• PathHandle only available for files
• Wrong FileSystem: " + stat.getPath()

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e1f6e9de7f8bef91. Report an issue: GitHub.