{"record":{"id":"096768b88f071b1a","repo":"apache/hadoop","slug":"end-of-file-reached-before-reading-fully-096768","errorCode":null,"errorMessage":"End of file reached before reading fully.","messagePattern":"End of file reached before reading fully\\.","errorType":"exception","errorClass":"EOFException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java","lineNumber":819,"sourceCode":"      throws IOException {\n    checkNotClosed();\n    validatePositionedReadArgs(position, buffer, offset, length);\n    getS3AStreamStatistics().readFullyOperationStarted(position, length);\n    if (length == 0) {\n      return;\n    }\n    int nread = 0;\n    synchronized (this) {\n      long oldPos = getPos();\n      try {\n        seek(position);\n        while (nread < length) {\n          int nbytes = read(buffer, offset + nread, length - nread);\n          if (nbytes < 0) {\n            // no attempt is currently made to recover from stream read problems;\n            // a lazy seek to the offset is probably the solution.\n            // but it will need more qualification against failure handling\n            throw new EOFException(FSExceptionMessages.EOF_IN_READ_FULLY);\n          }\n          nread += nbytes;\n        }\n      } finally {\n        seekQuietly(oldPos);\n      }\n    }\n  }\n\n  /**\n   * {@inheritDoc}\n   * Pass to {@link #readVectored(List, IntFunction, Consumer)}\n   * with the {@link VectoredReadUtils#LOG_BYTE_BUFFER_RELEASED} releaser.\n   * @param ranges the byte ranges to read.\n   * @param allocate the function to allocate ByteBuffer.\n   * @throws IOException IOE if any.\n   */\n  @Override","sourceCodeStart":801,"sourceCodeEnd":837,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java#L801-L837","documentation":"Thrown from S3AInputStream.readFully(position, buffer, offset, length) when the underlying read() returns -1 before length bytes were transferred. The original position is restored via seekQuietly in the finally block, but the call still fails with FSExceptionMessages.EOF_IN_READ_FULLY: the object ended before the requested range was satisfied.","triggerScenarios":"readFully(position, buf, off, len) where position + len exceeds the object's length; reading a fixed-size header/footer from a file shorter than assumed; reading an object that was concurrently truncated or replaced with a smaller one.","commonSituations":"Columnar readers (ORC/Parquet/Avro) reading footers from tiny, empty, or truncated files; zero-byte uploads; ranges computed from a stale FileStatus while the object changed; split logic assuming a minimum file size.","solutions":["Check position + length against fs.getFileStatus(path).getLen() before readFully and treat short files as invalid/empty input","Catch EOFException and handle 'file too short' explicitly with a clearer domain error instead of letting it propagate raw","If concurrent writes are possible, verify the object was not truncated/overwritten between status and read","For footer formats, validate the magic/size fields against the actual file length before issuing the read"],"exampleFix":"// before\nin.readFully(pos, buf, 0, FOOTER_LEN); // EOF: End of file reached before reading fully\n\n// after\nlong remaining = fileStatus.getLen() - pos;\nif (remaining < FOOTER_LEN) {\n  throw new IOException(\"File \" + path + \" too short: need \" + FOOTER_LEN\n      + \" bytes at \" + pos + \", only \" + remaining + \" available\");\n}\nin.readFully(pos, buf, 0, FOOTER_LEN);","handlingStrategy":"validation","validationCode":"long remaining = fs.getFileStatus(path).getLen() - position;\nif (remaining < length) {\n  throw new IOException(\"Requested \" + length + \" bytes at \" + position\n      + \" but only \" + Math.max(remaining, 0) + \" remain in \" + path);\n}\nin.readFully(position, buffer, offset, length);","typeGuard":null,"tryCatchPattern":"try { in.readFully(pos, buf, off, len); } catch (EOFException e) { /* file is shorter than assumed - fail with a domain error or retry with a smaller read; do not blindly retry the same length */ }","preventionTips":["Always compare position+length against the authoritative file length before readFully","Treat 0-byte and tiny files as an explicit case before footer parsing","With concurrent writers, read from a consistent snapshot or verify lengths immediately before the read"],"tags":["s3a","hadoop-aws","eofexception","readfully","truncated-file","footer-read"],"backgroundTag":"eof-during-read","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}