{"record":{"id":"cf1fd72447872702","repo":"apache/iceberg","slug":"ignoring-filenotfoundexception-when-listing-partit-cf1fd7","errorCode":null,"errorMessage":"Ignoring FileNotFoundException when listing partition of {}","messagePattern":"Ignoring FileNotFoundException when listing partition of (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java","lineNumber":184,"sourceCode":"      PartitionSpec spec,\n      SerializableConfiguration conf,\n      MetricsConfig metricsConfig,\n      NameMapping mapping,\n      boolean ignoreMissingFiles,\n      ExecutorService service) {\n    try {\n      return TableMigrationUtil.listPartition(\n          partition.values,\n          partition.uri,\n          partition.format,\n          spec,\n          conf.get(),\n          metricsConfig,\n          mapping,\n          service);\n    } catch (RuntimeException e) {\n      if (ignoreMissingFiles && e.getCause() instanceof FileNotFoundException) {\n        LOG.warn(\"Ignoring FileNotFoundException when listing partition of {}\", partition, e);\n        return Collections.emptyList();\n      } else {\n        throw e;\n      }\n    }\n  }\n\n  private static SparkPartition toSparkPartition(\n      CatalogTablePartition partition, CatalogTable table) {\n    Option<URI> locationUri = partition.storage().locationUri();\n    Option<String> partitionSerde = partition.storage().serde();\n\n    Preconditions.checkArgument(locationUri.nonEmpty(), \"Partition URI should be defined\");\n    Preconditions.checkArgument(\n        partitionSerde.nonEmpty() || table.provider().nonEmpty(),\n        \"Partition format should be defined\");\n\n    String uri = Util.uriToString(locationUri.get());","sourceCodeStart":166,"sourceCodeEnd":202,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java#L166-L202","documentation":"When listing a Hive partition's files during SparkTableUtil.listPartition, a FileNotFoundException wrapped in a RuntimeException can occur if the partition directory/files disappeared between metadata listing and file listing. If ignoreMissingFiles is set, the warning is logged and an empty list is returned instead of failing the import/migration.","triggerScenarios":"Migrating or importing a Hive table where a partition's underlying files were deleted (or not yet written) after the metastore recorded the partition, with spark.sql.iceberg.ignore-missing-files / ignoreMissingFiles=true.","commonSituations":"Racing jobs that drop or overwrite Hive partitions during a migration; partitions whose data was manually removed from storage; HDFS/S3 eventual consistency issues.","solutions":["Set ignore-missing-files to true if missing partition data is acceptable, then re-check counts after migration","Restore/recreate the missing partition data in the source table before re-running the import","Verify partition locations with a filesystem listing to find which partitions are actually gone"],"exampleFix":"// before\nspark.sql.iceberg.ignore-missing-files=false\n// after\nspark.sql.iceberg.ignore-missing-files=true","handlingStrategy":"fallback","validationCode":"// before migration, confirm partition locations exist\nfor (String loc : partitionLocations) {\n  if (!fs.exists(new Path(loc))) missing.add(loc);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Quiesce writers/deleters on the Hive table during migration","Set ignore-missing-files=true only when partial import is acceptable","Reconcile row counts post-import"],"tags":["spark","hive-migration","file-not-found","partitioning"],"backgroundTag":"file-not-found","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}