apache/iceberg · error · EOFException

Reached the end of stream with

Error message

Reached the end of stream with <remaining> bytes left to read

What it means

IOUtil.readFully() promises to fill the requested byte range from an InputStream in as many read() calls as needed. If the stream hits EOF before all requested bytes were read, it throws this EOFException with the number of bytes still missing. It is a data-truncation signal: the underlying file/stream is shorter than the read range requested.

Solutions

1. Check the file exists and its actual length covers fileOffset+length; re-copy/re-upload the file if truncated
2. Invalidate stale caches/metadata (content length, footer offsets) and re-plan the read
3. Verify the reader version and writer version are compatible — files written by mismatched formats may carry lengths the reader misinterprets
4. If EOF is legitimately possible, use partial reads (InputStream.read in a loop / RangeReadable.read) instead of readFully and handle short reads

Example fix

// before
IOUtil.readFully(inputFile, footerOffset, footerBytes, 0, footerBytes.length); // EOFException on truncated file
// after
if (inputFile.getLength() < footerOffset + footerBytes.length) {
  throw new IOException("File too short for footer: " + inputFile.getLocation());
}
IOUtil.readFully(inputFile, footerOffset, footerBytes, 0, footerBytes.length);

if (inputFile.getLength() < fileOffset + length) {
  throw new IOException("Read range exceeds file length " + inputFile.getLength()
      + ": offset=" + fileOffset + " len=" + length);
}
IOUtil.readFully(inputFile, fileOffset, bytes, offset, length);

try {
  IOUtil.readFully(inputFile, offset, bytes, 0, length);
} catch (EOFException e) {
  throw new IOException("Truncated file: " + inputFile.getLocation(), e);
}

When it happens

Trigger: Calling IOUtil.readFully(stream, bytes, off, length) or readFully(inputFile, fileOffset, ...) when the input file/stream has fewer than length bytes remaining from the current position (e.g. reading a footer at an offset beyond a truncated file, or a stale length metadata entry).

Common situations: Reading Parquet/ORAv1 footers of files truncated by a failed upload or partial copy; a cached content-length (S3 metadata) that no longer matches the object; a corrupted manifest pointing past EOF; reading a file concurrently overwritten with a smaller version.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Cannot seek to position after end of file
• could not read page in col " + desc
• could not read page in col
• could not read page " + valueCount + " in col " + desc
• Error reading mini block.

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/cdc98bb69dd0ffbe. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/io/IOUtil.java:47

  private IOUtil() {}

  private static final int WRITE_CHUNK_SIZE = 8192;

  /**
   * Reads into a buffer from a stream, making multiple read calls if necessary.
   *
   * @param stream an InputStream to read from
   * @param bytes a buffer to write into
   * @param offset starting offset in the buffer for the data
   * @param length length of bytes to copy from the input stream to the buffer
   * @throws EOFException if the end of the stream is reached before reading length bytes
   * @throws IOException if there is an error while reading
   */
  public static void readFully(InputStream stream, byte[] bytes, int offset, int length)
      throws IOException {
    int bytesRead = readRemaining(stream, bytes, offset, length);
    if (bytesRead < length) {
      throw new EOFException(
          "Reached the end of stream with " + (length - bytesRead) + " bytes left to read");
    }
  }

  /**
   * Reads exactly {@code length} bytes from the input file starting at {@code fileOffset} into the
   * buffer. Uses range reads when supported.
   *
   * @param inputFile the file to read from
   * @param fileOffset the position in the file to start reading from
   * @param bytes a buffer to write into
   * @param offset starting offset in the buffer for the data
   * @param length number of bytes to read
   * @throws IOException if there is an error while reading or if the end of the stream is reached
   *     before reading length bytes
   */
  public static void readFully(
      InputFile inputFile, long fileOffset, byte[] bytes, int offset, int length)

View on GitHub (pinned to 86d9c8fc54)