apache/iceberg · error · EOFException

Reached the end of stream with X bytes left to read

Error message

Reached the end of stream with X bytes left to read

What it means

GCSInputStream.readFully(position, buffer, offset, length) reads an exact byte range from a GCS ReadChannel. If fewer than the requested bytes are returned before the stream ends, it throws EOFException('Reached the end of stream with X bytes left to read'), indicating a truncated object or a read range beyond the object's size.

Source

Thrown at gcp/src/main/java/org/apache/iceberg/gcp/gcs/GCSInputStream.java:168

    int bytesRead = read(channel, byteBuffer, off, len);
    if (bytesRead == -1) {
      return -1;
    }

    pos += bytesRead;
    readBytes.increment(bytesRead);
    readOperations.increment();
    return bytesRead;
  }

  @Override
  public void readFully(long position, byte[] buffer, int offset, int length) throws IOException {
    try (ReadChannel readChannel = openChannel()) {
      readChannel.seek(position);
      readChannel.limit(position + length);
      int bytesRead = read(readChannel, ByteBuffer.wrap(buffer), offset, length);
      if (bytesRead < length) {
        throw new EOFException(
            "Reached the end of stream with " + (length - bytesRead) + " bytes left to read");
      }
      if (bytesRead > 0) {
        readBytes.increment(bytesRead);
        readOperations.increment();
      }
    }
  }

  @Override
  public int readTail(byte[] buffer, int offset, int length) throws IOException {
    if (blobSize == null) {
      blobSize = storage.get(blobId).getSize();
    }
    long startPosition = Math.max(0, blobSize - length);
    try (ReadChannel readChannel = openChannel()) {
      readChannel.seek(startPosition);
      int bytesRead = read(readChannel, ByteBuffer.wrap(buffer), offset, length);

View on GitHub (pinned to 86d9c8fc54)

Solutions

Verify the object size matches the length being requested — refresh file metadata if the object may have been replaced
Check for concurrent overwrite/delete of the object while reading; use generation-consistent reads
Fix offset arithmetic in the caller so ranges stay within the object bounds
Wrap in try-catch for EOFException and re-plan/refresh the scan rather than retrying blindly

Example fix

// before
in.readFully(position, buffer, offset, length); // may pass EOF
// after
long available = objectSize - position;
if (length > available) {
  throw new IOException("Range exceeds object size: " + position + "+" + length + " > " + objectSize);
}
in.readFully(position, buffer, offset, length);

Defensive patterns

Strategy: try-catch

Validate before calling

if (position + length > fileSize) { throw new IOException("Read range past EOF: " + position + "+" + length + " > " + fileSize); }

Try / catch

try { in.readFully(position, buffer, offset, length); } catch (EOFException e) { LOG.error("Truncated GCS read at {}", position, e); /* refresh metadata or re-plan */ }

Prevention

Refresh file metadata if objects may be replaced concurrently
Compute ranges from the actual object size, not cached lengths
Detect partially uploaded objects before reading

When it happens

Trigger: readFully requests a range whose end exceeds the object's actual size, or the ReadChannel returns a short read due to truncation/corruption, called internally by decode-style readers.

Common situations: Stale length metadata (file shrunk/replaced between listing and read), reading past the end of a file after a bad offset calculation, or partially uploaded objects.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/fbc19b6ac3e6319a. Report an issue: GitHub.