{"record":{"id":"2d9120a19af5e313","repo":"apache/iceberg","slug":"unable-to-read-avro-file","errorCode":null,"errorMessage":"Unable to read Avro file: ","messagePattern":"Unable to read Avro file: ","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"data/src/main/java/org/apache/iceberg/data/TableMigrationUtil.java","lineNumber":220,"sourceCode":"        throw new UnsupportedOperationException(\"Unknown partition format: \" + format);\n      }\n      return Arrays.asList(datafiles);\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Unable to list files in partition: \" + partitionUri, e);\n    } finally {\n      if (service != null) {\n        service.shutdown();\n      }\n    }\n  }\n\n  private static Metrics getAvroMetrics(Path path, Configuration conf) {\n    try {\n      InputFile file = HadoopInputFile.fromPath(path, conf);\n      long rowCount = Avro.rowCount(file);\n      return new Metrics(rowCount, null, null, null, null);\n    } catch (UncheckedIOException e) {\n      throw new RuntimeException(\"Unable to read Avro file: \" + path, e);\n    }\n  }\n\n  private static Metrics getParquetMetrics(\n      Path path, Configuration conf, MetricsConfig metricsSpec, NameMapping mapping) {\n    try {\n      InputFile file = HadoopInputFile.fromPath(path, conf);\n      return ParquetUtil.fileMetrics(file, metricsSpec, mapping);\n    } catch (UncheckedIOException e) {\n      throw new RuntimeException(\"Unable to read the metrics of the Parquet file: \" + path, e);\n    }\n  }\n\n  private static Metrics getOrcMetrics(\n      Path path, Configuration conf, MetricsConfig metricsSpec, NameMapping mapping) {\n    try {\n      return OrcMetrics.fromInputFile(HadoopInputFile.fromPath(path, conf), metricsSpec, mapping);\n    } catch (UncheckedIOException e) {","sourceCodeStart":202,"sourceCodeEnd":238,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/data/src/main/java/org/apache/iceberg/data/TableMigrationUtil.java#L202-L238","documentation":"TableMigrationUtil.getAvroMetrics reads an Avro file's row count via HadoopInputFile and Avro.rowCount. If reading throws UncheckedIOException (unreadable/corrupt file, IO error), it is rethrown as a RuntimeException 'Unable to read Avro file: <path>'. Only the row count metric is computed; the failure means the file could not be opened/read at all.","triggerScenarios":"Calling metrics()/listPartition on an Avro data file that is corrupt, truncated, missing, or unreadable due to permissions/filesystem errors — Avro.rowCount's UncheckedIOException is converted here.","commonSituations":"Migrating legacy Avro tables with damaged or partially-written files; files deleted between listing and metric computation; permissions changed mid-migration; HDFS/S3 transient errors while reading the footer.","solutions":["Open the chained cause and check the file directly (permissions, existence, footer integrity).","Repair or re-export the corrupt/truncated Avro file before migrating.","Exclude the damaged file from migration and re-add it after repair.","Retry on transient storage errors (throttling, connection resets)."],"exampleFix":"// before: one bad Avro file aborts metrics for the partition\nMetrics m = TableMigrationUtil.metrics(path, conf, spec, \"avro\", metricsSpec, mapping);\n\n// after: pre-check readability\nif (fs.exists(path) && fs.open(path) != null) {\n  Metrics m = TableMigrationUtil.metrics(path, conf, spec, \"avro\", metricsSpec, mapping);\n}","handlingStrategy":"validation","validationCode":"// pre-check the Avro file is readable before computing metrics\ntry (DataFileReader<Object, Object> r =\n         new DataFileReader<>(new File(path.toUri()), new GenericDatumReader<>())) {\n  r.getBlockCount();\n}","typeGuard":null,"tryCatchPattern":"try {\n  Metrics m = TableMigrationUtil.metrics(path, conf, spec, \"avro\", metricsSpec, mapping);\n} catch (RuntimeException e) {\n  if (e.getMessage().startsWith(\"Unable to read Avro file\")) {\n    quarantine(path); // repair or exclude the corrupt file\n  } else { throw e; }\n}","preventionTips":["Checksum/validate legacy Avro files before migration.","Re-run listings close to metric computation to avoid files deleted in between.","Quarantine-and-skip corrupt files instead of aborting the whole migration."],"tags":["avro","io","migration","metrics","corrupt-file"],"backgroundTag":"file-read-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}