{"record":{"id":"cbfa236d17501359","repo":"apache/iceberg","slug":"unable-to-list-files-in-partition","errorCode":null,"errorMessage":"Unable to list files in partition: ","messagePattern":"Unable to list files in partition: ","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"data/src/main/java/org/apache/iceberg/data/TableMigrationUtil.java","lineNumber":206,"sourceCode":"              Metrics metrics =\n                  getParquetMetrics(fileStatus.get(index).getPath(), conf, metricsSpec, mapping);\n              datafiles[index] =\n                  buildDataFile(fileStatus.get(index), partitionValues, spec, metrics, \"parquet\");\n            });\n      } else if (format.contains(\"orc\")) {\n        task.run(\n            index -> {\n              Metrics metrics =\n                  getOrcMetrics(fileStatus.get(index).getPath(), conf, metricsSpec, mapping);\n              datafiles[index] =\n                  buildDataFile(fileStatus.get(index), partitionValues, spec, metrics, \"orc\");\n            });\n      } else {\n        throw new UnsupportedOperationException(\"Unknown partition format: \" + format);\n      }\n      return Arrays.asList(datafiles);\n    } catch (IOException e) {\n      throw new UncheckedIOException(\"Unable to list files in partition: \" + partitionUri, e);\n    } finally {\n      if (service != null) {\n        service.shutdown();\n      }\n    }\n  }\n\n  private static Metrics getAvroMetrics(Path path, Configuration conf) {\n    try {\n      InputFile file = HadoopInputFile.fromPath(path, conf);\n      long rowCount = Avro.rowCount(file);\n      return new Metrics(rowCount, null, null, null, null);\n    } catch (UncheckedIOException e) {\n      throw new RuntimeException(\"Unable to read Avro file: \" + path, e);\n    }\n  }\n\n  private static Metrics getParquetMetrics(","sourceCodeStart":188,"sourceCodeEnd":224,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/data/src/main/java/org/apache/iceberg/data/TableMigrationUtil.java#L188-L224","documentation":"TableMigrationUtil.listPartition wraps any IOException thrown while listing the partition's files (filesystem listing, parallel metric computation) into an UncheckedIOException 'Unable to list files in partition: <partitionUri>'. It indicates the partition directory could not be read, not a format problem.","triggerScenarios":"Calling listPartition for a partition URI that doesn't exist, is inaccessible (permissions/auth), or whose underlying filesystem errors during listing or during getOrcMetrics/getParquetMetrics IO work inside the Tasks.foreach.","commonSituations":"HDFS/S3 outages or throttling during migration; wrong partition path in the source table metadata; expired cloud credentials; missing Kerberos/permissions on the Hive warehouse directory.","solutions":["Read the chained cause to find whether it's a FileNotFoundException, access denied, or connection error.","Verify the partition URI exists and is correct (e.g. hadoop fs -ls <partitionUri>).","Fix filesystem credentials/permissions for the migration job's identity.","Retry if the cause indicates a transient storage error; exclude corrupt partitions otherwise."],"exampleFix":"// before: migration fails wholesale on one bad partition\nList<DataFile> files = TableMigrationUtil.listPartition(...);\n\n// after: validate and skip missing partitions first\nif (fs.exists(partitionPath)) {\n  List<DataFile> files = TableMigrationUtil.listPartition(...);\n}","handlingStrategy":"validation","validationCode":"if (!fs.exists(new Path(partitionUri))) {\n  throw new IllegalArgumentException(\"Partition does not exist: \" + partitionUri);\n}\nfs.access(new Path(partitionUri), FsAction.READ);","typeGuard":null,"tryCatchPattern":"try {\n  List<DataFile> files = TableMigrationUtil.listPartition(...);\n} catch (UncheckedIOException e) {\n  if (e.getCause() instanceof FileNotFoundException) { skipPartition(); }\n  else if (isTransient(e.getCause())) { retry(); }\n  else { throw e; }\n}","preventionTips":["Verify partition URIs in source table metadata before migration.","Grant the migration job's principal read access to the warehouse path.","Retry transient storage errors; use fresh, valid cloud credentials."],"tags":["io","migration","filesystem","unchecked-ioexception","listing"],"backgroundTag":"file-read-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}