apache/iceberg · error · RuntimeIOException

Failed to read metadata for file

Error message

Failed to read metadata for file: %s

What it means

AvroIterable.getMetadata lazily opens the Avro file solely to read the footer/metadata map. If opening or reading the footer raises an IOException (the file is missing, truncated, or not a valid Avro container), the exception is wrapped in a RuntimeIOException that includes the file location.

Solutions

1. Check the file exists and is readable at the given location (list/stat via FileIO)
2. Confirm the file is actually an Avro container and the read uses the matching Avro init/model (e.g. GenericAvroReader) and compression codec is on the classpath (zstd/bzip2/snappy)
3. Re-resolve the path from current table metadata in case the file was replaced or deleted
4. Fix storage credentials/permissions so the reader can open the object

Example fix

// before: hardcoded stale path
AvroIterable<Record> it = Avro.read(io().newInputFile("s3://bucket/old/part-0000.avro")).build();
Map<String,String> meta = it.getMetadata();
// after: resolve from current manifest entry
DataFile df = currentManifestRow(dataFilePath);
AvroIterable<Record> it = Avro.read(io().newInputFile(df.location())).build();
Map<String,String> meta = it.getMetadata();

boolean readable = fileIO.exists(location) && fileIO.newInputFile(location).getLength() > 0;
if (!readable) throw new IllegalStateException("File missing or empty: " + location);

try (AvroIterable<D> it = Avro.read(in).project(schema).build()) {
  Map<String,String> meta = it.getMetadata();
} catch (RuntimeIOException e) {
  LOG.error("Cannot read Avro metadata for {}", location, e);
  throw new UncheckedIOException(e);
}

When it happens

Trigger: Calling AvroIterable.getMetadata() (or any access path that initializes metadata) when the file at file.location() cannot be opened or its Avro footer cannot be parsed: key-decompression mismatch, wrong codec, zero-byte file, or deleted object.

Common situations: Pointing a reader at a file deleted by a concurrent compaction/expiry; uploading an incomplete file; trying to read an Orc/Parquet file with the Avro reader by mistake; permissions blocking the read on HDFS/S3.

Understand the failure class

Background: "open() failed", "failed to open file", "cannot create file" — what a file open error means and how to fix it — this error's family across 42 libraries.

Related errors

• Cannot read manifest list file
• Cannot read manifest list file
• Decoding datum failed
• Failed to check range end
• Failed to close manifest reader

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/ffac3a4269b4ef68. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/avro/AvroIterable.java:70

    this.reuseContainers = reuseContainers;
  }

  private DataFileReader<D> initMetadata(DataFileReader<D> metadataReader) {
    if (metadata == null) {
      this.metadata = Maps.newHashMap();
      for (String key : metadataReader.getMetaKeys()) {
        metadata.put(key, metadataReader.getMetaString(key));
      }
    }
    return metadataReader;
  }

  public Map<String, String> getMetadata() {
    if (metadata == null) {
      try (DataFileReader<D> reader = newFileReader()) {
        initMetadata(reader);
      } catch (IOException e) {
        throw new RuntimeIOException(e, "Failed to read metadata for file: %s", file.location());
      }
    }
    return metadata;
  }

  @Override
  public CloseableIterator<D> iterator() {
    FileReader<D> fileReader = initMetadata(newFileReader());

    if (start != null) {
      if (reader instanceof SupportsRowPosition) {
        ((SupportsRowPosition) reader)
            .setRowPositionSupplier(
                Suppliers.memoize(() -> AvroIO.findStartingRowPos(file::newStream, start)));
      }
      fileReader = new AvroRangeIterator<>(fileReader, start, end);
    } else if (reader instanceof SupportsRowPosition) {
      ((SupportsRowPosition) reader).setRowPositionSupplier(() -> 0L);

View on GitHub (pinned to 86d9c8fc54)