{"record":{"id":"21ddb40f8f01e911","repo":"apache/iceberg","slug":"failed-to-validate-no-deleted-data-files-matching","errorCode":null,"errorMessage":"Failed to validate no deleted data files matching %s","messagePattern":"Failed to validate no deleted data files matching (.+?)","errorType":"exception","errorClass":"UncheckedIOException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java","lineNumber":695,"sourceCode":"   * @param dataFilter an expression used to find deleted data files\n   * @param parent ending snapshot on the branch being validated\n   */\n  protected void validateDeletedDataFiles(\n      TableMetadata base, Long startingSnapshotId, Expression dataFilter, Snapshot parent) {\n    CloseableIterable<ManifestEntry<DataFile>> conflictEntries =\n        deletedDataFiles(base, startingSnapshotId, dataFilter, null, parent);\n\n    try (CloseableIterator<ManifestEntry<DataFile>> conflicts = conflictEntries.iterator()) {\n      if (conflicts.hasNext()) {\n        throw new ValidationException(\n            \"Found conflicting deleted files that can contain records matching %s: %s\",\n            dataFilter,\n            Iterators.toString(\n                Iterators.transform(conflicts, entry -> entry.file().location().toString())));\n      }\n\n    } catch (IOException e) {\n      throw new UncheckedIOException(\n          String.format(\"Failed to validate no deleted data files matching %s\", dataFilter), e);\n    }\n  }\n\n  /**\n   * Validates that no files matching a filter have been deleted from the table since a starting\n   * snapshot.\n   *\n   * @param base table metadata to validate\n   * @param startingSnapshotId id of the snapshot current at the start of the operation\n   * @param partitionSet a partition set used to find deleted data files\n   * @param parent ending snapshot on the branch being validated\n   */\n  protected void validateDeletedDataFiles(\n      TableMetadata base, Long startingSnapshotId, PartitionSet partitionSet, Snapshot parent) {\n    CloseableIterable<ManifestEntry<DataFile>> conflictEntries =\n        deletedDataFiles(base, startingSnapshotId, null, partitionSet, parent);\n","sourceCodeStart":677,"sourceCodeEnd":713,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java#L677-L713","documentation":"UncheckedIOException wrapping an IOException that occurred while scanning manifests to verify no concurrently deleted data files match the given filter. The delete-conflict validation could not complete because manifest data could not be read, so the commit is aborted. The real cause is available on the chained exception.","triggerScenarios":"Calling validateNoDeletedFiles (filter variant) during a commit where iterating deletedDataFiles manifest entries throws IOException from FileIO (network/storage failure).","commonSituations":"S3 503 slowdowns or credential expiry during commit validation; HDFS/DataNode failures; very long validation windows increasing exposure to transient storage faults.","solutions":["Retry the commit with a retry policy (Tasks.foreach with exponential backoff) once storage is healthy.","Diagnose and fix the underlying IOException cause (connectivity, credentials, permissions) from the exception chain.","Check that manifest files for the validation start snapshot still exist and are readable.","Shorten the time between refresh and commit to shrink the manifest-scan window."],"exampleFix":"// before\ntable.newOverwrite().validateNoDeletedFiles(oldSnapshotId).commit(); // UncheckedIOException on transient S3 error\n// after\nTasks.foreach(() -> {\n      table.refresh();\n      table.newOverwrite().validateNoDeletedFiles(table.currentSnapshot().snapshotId()).commit();\n    })\n    .retry(3).exponentialBackoff(200, 10000, 60000)\n    .run();","handlingStrategy":"retry","validationCode":"// pre-check manifest readability before commit\nmanifestsTable.entries(); // touch validation inputs via a metadata table read","typeGuard":null,"tryCatchPattern":"try {\n  operation.commit();\n} catch (UncheckedIOException e) {\n  // fix underlying IOException cause, then retry\n}","preventionTips":["Monitor storage backend health (S3 throttling, HDFS) around commit windows","Retry commits with backoff","Refresh shortly before commit to shorten validation scans"],"tags":["io","concurrency","commit-validation","unchecked-io-exception"],"backgroundTag":"file-read-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}