What causes "Failed to check range end: %d" in apache/iceberg?

Retry the task; transient IO failures during iteration are typically resolved with a fresh read

apache/iceberg · error · RuntimeIOException

Failed to check range end: %d

Error message

Failed to check range end: %d

What it means

AvroRangeIterator.hasNext() checks both that the underlying Avro reader has more records and that it has not passed the configured end offset via reader.pastSync(end). An IOException from either call is wrapped in RuntimeIOException reporting the range end, since the iterator cannot determine whether the range is complete.

Source

Thrown at core/src/main/java/org/apache/iceberg/avro/AvroIterable.java:142

      try {
        reader.sync(start);
      } catch (IOException e) {
        throw new RuntimeIOException(e, "Failed to find sync past position %d", start);
      }
    }

    @Override
    public Schema getSchema() {
      return reader.getSchema();
    }

    @Override
    public boolean hasNext() {
      try {
        return reader.hasNext() && !reader.pastSync(end);
      } catch (IOException e) {
        throw new RuntimeIOException(e, "Failed to check range end: %d", end);
      }
    }

    @Override
    public D next() {
      if (!hasNext()) {
        throw new NoSuchElementException();
      }
      return reader.next();
    }

    @Override
    public D next(D reuse) {
      if (!hasNext()) {
        throw new NoSuchElementException();
      }
      try {
        return reader.next(reuse);

View on GitHub (pinned to 86d9c8fc54)

Solutions

Retry the task; transient IO failures during iteration are typically resolved with a fresh read
Re-open the data file with a fresh AvroIterable and skip already-consumed records
Ensure the file is not concurrently rewritten (isolate compaction from reads via snapshot isolation)
Check storage/network stability (timeouts, proxies) for large files

Defensive patterns

Strategy: try-catch

Validate before calling

// ensure range end does not exceed the file length
Preconditions.checkArgument(end <= inputFile.getLength(), "Range end %s exceeds file length", end);

Try / catch

try (CloseableIterator<D> iter = iterable.iterator()) {
  while (iter.hasNext()) consume(iter.next());
} catch (RuntimeIOException e) {
  retryWithFreshHandle(remaining -> ...);
}

Prevention

Set robust object-store timeouts/retries for long scans
Avoid modifying/compacting files while tasks read them
Checkpoint progress so tasks can resume from a known offset after failure
Monitor network stability in clusters reading large remote files

When it happens

Trigger: Iterating a byte-range AvroIterable when the underlying stream fails during hasNext()/pastSync: object store connection reset, truncated block stream, or a file modified while being read so pastSync cannot be evaluated.

Common situations: Long-running Spark/Flink tasks reading large Avro files whose network connections time out; preemption of the underlying stream; file overwritten concurrently by compaction while the range iterator is open.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/48dbecb357dd84a2. Report an issue: GitHub.