{"record":{"id":"cdc98bb69dd0ffbe","repo":"apache/iceberg","slug":"reached-the-end-of-stream-with-remaining-bytes-l","errorCode":null,"errorMessage":"Reached the end of stream with <remaining> bytes left to read","messagePattern":"Reached the end of stream with <remaining> bytes left to read","errorType":"exception","errorClass":"EOFException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/org/apache/iceberg/io/IOUtil.java","lineNumber":47,"sourceCode":"  private IOUtil() {}\n\n  private static final int WRITE_CHUNK_SIZE = 8192;\n\n  /**\n   * Reads into a buffer from a stream, making multiple read calls if necessary.\n   *\n   * @param stream an InputStream to read from\n   * @param bytes a buffer to write into\n   * @param offset starting offset in the buffer for the data\n   * @param length length of bytes to copy from the input stream to the buffer\n   * @throws EOFException if the end of the stream is reached before reading length bytes\n   * @throws IOException if there is an error while reading\n   */\n  public static void readFully(InputStream stream, byte[] bytes, int offset, int length)\n      throws IOException {\n    int bytesRead = readRemaining(stream, bytes, offset, length);\n    if (bytesRead < length) {\n      throw new EOFException(\n          \"Reached the end of stream with \" + (length - bytesRead) + \" bytes left to read\");\n    }\n  }\n\n  /**\n   * Reads exactly {@code length} bytes from the input file starting at {@code fileOffset} into the\n   * buffer. Uses range reads when supported.\n   *\n   * @param inputFile the file to read from\n   * @param fileOffset the position in the file to start reading from\n   * @param bytes a buffer to write into\n   * @param offset starting offset in the buffer for the data\n   * @param length number of bytes to read\n   * @throws IOException if there is an error while reading or if the end of the stream is reached\n   *     before reading length bytes\n   */\n  public static void readFully(\n      InputFile inputFile, long fileOffset, byte[] bytes, int offset, int length)","sourceCodeStart":29,"sourceCodeEnd":65,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/io/IOUtil.java#L29-L65","documentation":"IOUtil.readFully() promises to fill the requested byte range from an InputStream in as many read() calls as needed. If the stream hits EOF before all requested bytes were read, it throws this EOFException with the number of bytes still missing. It is a data-truncation signal: the underlying file/stream is shorter than the read range requested.","triggerScenarios":"Calling IOUtil.readFully(stream, bytes, off, length) or readFully(inputFile, fileOffset, ...) when the input file/stream has fewer than length bytes remaining from the current position (e.g. reading a footer at an offset beyond a truncated file, or a stale length metadata entry).","commonSituations":"Reading Parquet/ORAv1 footers of files truncated by a failed upload or partial copy; a cached content-length (S3 metadata) that no longer matches the object; a corrupted manifest pointing past EOF; reading a file concurrently overwritten with a smaller version.","solutions":["Check the file exists and its actual length covers fileOffset+length; re-copy/re-upload the file if truncated","Invalidate stale caches/metadata (content length, footer offsets) and re-plan the read","Verify the reader version and writer version are compatible — files written by mismatched formats may carry lengths the reader misinterprets","If EOF is legitimately possible, use partial reads (InputStream.read in a loop / RangeReadable.read) instead of readFully and handle short reads"],"exampleFix":"// before\nIOUtil.readFully(inputFile, footerOffset, footerBytes, 0, footerBytes.length); // EOFException on truncated file\n// after\nif (inputFile.getLength() < footerOffset + footerBytes.length) {\n  throw new IOException(\"File too short for footer: \" + inputFile.getLocation());\n}\nIOUtil.readFully(inputFile, footerOffset, footerBytes, 0, footerBytes.length);","handlingStrategy":"validation","validationCode":"if (inputFile.getLength() < fileOffset + length) {\n  throw new IOException(\"Read range exceeds file length \" + inputFile.getLength()\n      + \": offset=\" + fileOffset + \" len=\" + length);\n}\nIOUtil.readFully(inputFile, fileOffset, bytes, offset, length);","typeGuard":null,"tryCatchPattern":"try {\n  IOUtil.readFully(inputFile, offset, bytes, 0, length);\n} catch (EOFException e) {\n  throw new IOException(\"Truncated file: \" + inputFile.getLocation(), e);\n}","preventionTips":["Verify file length covers every offset+length you read","Avoid caching content lengths across overwrites; re-fetch length per file version","Validate checksums to catch truncated uploads early","Use RangeReadable when EOF is an expected possibility"],"tags":["io","eof","file-truncated","parquet"],"backgroundTag":"file-read-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}