{"record":{"id":"387ef8d2e3e804f5","repo":"apache/hadoop","slug":"can-t-read-filestatusproto-with-negative-size-of","errorCode":null,"errorMessage":"Can't read FileStatusProto with negative size of ${size}","messagePattern":"Can't read FileStatusProto with negative size of (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java","lineNumber":501,"sourceCode":"    sb.append(\"; hasAcl=\" + hasAcl())\n        .append(\"; isEncrypted=\" + isEncrypted())\n        .append(\"; isErasureCoded=\" + isErasureCoded())\n        .append(\"}\");\n    return sb.toString();\n  }\n\n  /**\n   * Read instance encoded as protobuf from stream.\n   * @param in Input stream\n   * @see PBHelper#convert(FileStatus)\n   * @deprecated Use the {@link PBHelper} and protobuf serialization directly.\n   */\n  @Override\n  @Deprecated\n  public void readFields(DataInput in) throws IOException {\n    int size = in.readInt();\n    if (size < 0) {\n      throw new IOException(\"Can't read FileStatusProto with negative \" +\n          \"size of \" + size);\n    }\n    byte[] buf = new byte[size];\n    in.readFully(buf);\n    FileStatusProto proto = FileStatusProto.parseFrom(buf);\n    FileStatus other = PBHelper.convert(proto);\n    isdir = other.isDirectory();\n    length = other.getLen();\n    block_replication = other.getReplication();\n    blocksize = other.getBlockSize();\n    modification_time = other.getModificationTime();\n    access_time = other.getAccessTime();\n    setPermission(other.getPermission());\n    setOwner(other.getOwner());\n    setGroup(other.getGroup());\n    setSymlink((other.isSymlink() ? other.getSymlink() : null));\n    setPath(other.getPath());\n    attr = attributes(other.hasAcl(), other.isEncrypted(),","sourceCodeStart":483,"sourceCodeEnd":519,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L483-L519","documentation":"FileStatus.readFields(DataInput) expects the wire format written by FileStatus.write(DataOutput): a 4-byte int size followed by exactly that many FileStatusProto bytes. If the size int decodes to a negative value the record cannot be valid, so the read aborts with IOException. A negative prefix almost always means the bytes were never in this format, or the stream position is misaligned after an earlier error.","triggerScenarios":"Invoking new FileStatus().readFields(in) (directly or through Writable-cloning/RPC machinery) on a DataInput whose next 4 bytes are not the length prefix: reading raw FileStatusProto bytes without the int prefix, continuing to read a stream after a prior short read or exception desynchronized the offset, or deserializing data written by an incompatible Hadoop version.","commonSituations":"Mixed Hadoop client/server versions in one pipeline; custom RPC or cache code hand-reading FileStatus bytes; truncated or partially-written sequence/RPC files; unit tests feeding arbitrary byte arrays into readFields.","solutions":["Confirm writer and reader run the same Hadoop version and that FileStatus.write (int size + protobuf bytes) actually produced the bytes being read","If the payload is raw protobuf, skip readFields and use FileStatusProto.parseFrom + PBHelper.convert","After any exception on a stream, do not keep reading at the current offset - reopen or resync before the next readFields","Hex-dump the first 8 bytes at the read position to verify the length prefix is a small positive int"],"exampleFix":"// before\nFileStatus st = new FileStatus();\nst.readFields(dataIn); // throws: next 4 bytes were not a size prefix\n\n// after: parse raw protobuf from an InputStream instead\nFileStatusProto proto = FileStatusProto.parseFrom(inputStream);\nFileStatus st = PBHelper.convert(proto);","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  status.readFields(in);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"negative\")) {\n    // stream misaligned or corrupt - do NOT retry on the same stream\n    in.close();\n    throw new IllegalStateException(\"Corrupt FileStatus stream at read offset\", e);\n  }\n  throw e;\n}","preventionTips":["Pin producer and consumer to the same hadoop-common version","Never interleave raw byte reads with readFields on the same DataInput","After any stream exception, reopen/resync instead of continuing to read","Prefer FileStatusProto.parseFull/PBHelper when you control both ends"],"tags":["hadoop","serialization","protobuf","filestatus","corruption"],"backgroundTag":"message-deserialization-corruption","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}