apache/iceberg · error · RuntimeIOException

Failed to open file: %s

Error message

Failed to open file: %s

What it means

AvroIterable.newFileReader opens the underlying InputFile and constructs the Avro DataFileReader. Any failure while creating the stream or initializing the Avro reader is wrapped in a RuntimeIOException naming the file location; the stream is closed before throwing.

Source

Thrown at core/src/main/java/org/apache/iceberg/avro/AvroIterable.java:113

    }

    return CloseableIterator.withClose(fileReader);
  }

  private DataFileReader<D> newFileReader() {
    SeekableInput stream = null;
    try {
      stream = AvroIO.stream(file.newStream(), file.getLength());
      return (DataFileReader<D>) DataFileReader.openReader(stream, reader);
    } catch (IOException e) {
      if (stream != null) {
        try {
          stream.close();
        } catch (IOException closeException) {
          // Ignore close exception
        }
      }
      throw new RuntimeIOException(e, "Failed to open file: %s", file.location());
    }
  }

  private static class AvroRangeIterator<D> implements FileReader<D> {
    private final FileReader<D> reader;
    private final long end;

    AvroRangeIterator(FileReader<D> reader, long start, long end) {
      this.reader = reader;
      this.end = end;

      try {
        reader.sync(start);
      } catch (IOException e) {
        throw new RuntimeIOException(e, "Failed to find sync past position %d", start);
      }
    }

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Verify storage credentials and endpoint configuration for the configured FileIO
2. Check the object exists and its size is > 0 (truncated/empty upload)
3. Add the missing codec dependency (e.g. aircompressor/zstd-jni) if the Avro codec is unsupported
4. Retry with a fresh InputFile to rule out transient network errors

Preconditions.checkNotNull(inputFile, "InputFile required");
if (inputFile.getLength() == 0) throw new IllegalStateException("Empty file: " + inputFile.location());

try (CloseableIterable<D> it = Avro.read(in).project(schema).build()) {
  it.forEach(consumer);
} catch (RuntimeIOException e) {
  throw new RuntimeException("Open failed for " + in.location(), e);
}

When it happens

Trigger: Calling getMetadata() or iterating (fileReader) when the InputFile stream() throws, the seekable stream cannot be created, or DataFileReader construction fails due to an invalid Avro magic header or unsupported codec.

Common situations: S3/GCS credentials missing or expired (403/404 surfaces as IOException); HDFS NameNode unavailable; file corrupted in transit; reading a file whose compression codec (e.g. zstd) library is absent.

Understand the failure class

Background: "open() failed", "failed to open file", "cannot create file" — what a file open error means and how to fix it — this error's family across 42 libraries.

Related errors

• Cannot read manifest list file: %s
• Failed to create snapshot list writer for path: %s
• Cannot read manifest list file: %s
• Failed to close manifest reader
• Failed to create manifest writer for path: %s

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/5ddd95cdf7f679d5. Report an issue: GitHub.